-- Logs begin at Sun 2022-08-07 18:37:48 UTC, end at Sun 2022-08-07 19:46:00 UTC. -- Aug 07 19:11:58.443024 dv-cinder-850320-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 07 19:11:58.454338 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 07 19:11:58.456348 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Aug 7 19:11:58 2022] *** Aug 07 19:11:58.456348 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 07 19:11:58.456348 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 07 19:11:58.456348 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: nodename: dv-cinder-850320-4-iscsi Aug 07 19:11:58.456348 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: machine: x86_64 Aug 07 19:11:58.456348 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: clock source: unix Aug 07 19:11:58.456348 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: pcre jit disabled Aug 07 19:11:58.456348 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: detected number of CPU cores: 4 Aug 07 19:11:58.456348 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: current working directory: / Aug 07 19:11:58.456348 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: detected binary path: /usr/bin/uwsgi-core Aug 07 19:11:58.456932 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: your processes number limit is 47680 Aug 07 19:11:58.456932 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: your memory page size is 4096 bytes Aug 07 19:11:58.456932 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: detected max file descriptor number: 2048 Aug 07 19:11:58.456932 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: lock engine: pthread robust mutexes Aug 07 19:11:58.456932 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: thunder lock: enabled Aug 07 19:11:58.456932 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 07 19:11:58.456932 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 07 19:11:58.494955 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: Python main interpreter initialized at 0x5610c5c9b9c0 Aug 07 19:11:58.494955 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: python threads support enabled Aug 07 19:11:58.494955 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: your server socket listen backlog is limited to 100 connections Aug 07 19:11:58.494955 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: your mercy for graceful operations on workers is 90 seconds Aug 07 19:11:58.495337 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: mapped 403077 bytes (393 KB) for 2 cores Aug 07 19:11:58.495407 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: *** Operational MODE: preforking *** Aug 07 19:11:58.495644 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: *** uWSGI is running in multiple interpreter mode *** Aug 07 19:11:58.495644 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: spawned uWSGI master process (pid: 80126) Aug 07 19:11:58.495834 dv-cinder-850320-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 07 19:11:58.498003 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: spawned uWSGI worker 1 (pid: 80138, cores: 1) Aug 07 19:11:58.498003 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: spawned uWSGI worker 2 (pid: 80139, cores: 1) Aug 07 19:11:58.498003 dv-cinder-850320-4-iscsi devstack@keystone.service[80126]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 07 19:12:03.901356 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 19:12:03.901356 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: from cryptography.utils import int_from_bytes Aug 07 19:12:03.902310 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 19:12:03.902310 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: from cryptography.utils import int_from_bytes Aug 07 19:12:03.929375 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 19:12:03.929375 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: from cryptography.utils import int_from_bytes Aug 07 19:12:03.930123 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 19:12:03.930123 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: from cryptography.utils import int_from_bytes Aug 07 19:12:03.951865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 07 19:12:03.952106 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 07 19:12:03.952303 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] command line args: [] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 07 19:12:03.952583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 07 19:12:03.952777 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ================================================================================ {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 07 19:12:03.953070 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] admin_token = **** {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.953354 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] config_dir = [] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.953572 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.953771 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] config_source = [] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.953978 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] debug = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.954193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.955374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.955374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.955374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.955374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.955757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] list_limit = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.955841 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] log_config_append = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.956043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.956237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] log_dir = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.956455 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] log_file = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.956722 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.957022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.957322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.957610 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.957790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.957965 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.958252 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.958521 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.958798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.959092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.959380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.959661 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.959952 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.960277 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.960659 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.961254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.961447 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] publish_errors = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.961734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.962012 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.962215 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.962556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.962842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.963156 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] standard_threads = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.963484 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.963781 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.963979 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.964201 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] use_journal = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.964402 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] use_json = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.964603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] use_stderr = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.964802 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] use_syslog = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.965083 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.965392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.965681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.966027 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.966392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.966726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.967043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.967361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.967681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.968051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.968504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.968854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.969089 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.969336 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.969574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.969819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.970069 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.970304 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.970558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.970836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.971052 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.971285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.971536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.971797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.972068 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.972318 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.972546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.972775 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.973002 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.973234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.973481 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.973732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.973960 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.974189 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.974433 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.974678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.974914 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.975161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.975391 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.975624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.975857 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.976105 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.976337 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.976566 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.976794 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.977027 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.977271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.977504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.977738 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.977966 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.978197 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.978430 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.978676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.978905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.979139 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.979373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.979607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.979840 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.980098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.980195 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 07 19:12:03.980328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.980413 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 07 19:12:03.980554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] command line args: [] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 07 19:12:03.980630 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.980796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.980880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 07 19:12:03.980995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ================================================================================ {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 07 19:12:03.981072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.981251 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] admin_token = **** {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.981329 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.981500 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] config_dir = [] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.981589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.981690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.981769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.981889 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] config_source = [] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.982071 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] debug = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.982156 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.982278 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.982391 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.982516 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.982601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.982696 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.982857 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.982967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.983216 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.983344 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.983449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] list_limit = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.983598 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.983687 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] log_config_append = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.983797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.983873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.983977 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] log_dir = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.984123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.984206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] log_file = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.984292 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.984413 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.984526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.984661 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.984772 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.984906 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.984996 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.985155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.985237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.985333 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.985453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.985539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.985691 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.985776 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.985926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.986008 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.986157 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.986284 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.986392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.986477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.986623 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.986707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.986858 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.986942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.987104 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.987219 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.987339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.987423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.987573 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.987668 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.987811 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.988074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.988155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.988293 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] publish_errors = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.988452 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.988528 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.988598 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.988745 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.988822 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.988922 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.989005 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.989168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.989246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.989401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.989484 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.989626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.989703 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] standard_threads = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.989853 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.989935 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.990084 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.990159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.990263 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.990340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.990440 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] use_journal = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.990557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.990635 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] use_json = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.990780 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] use_stderr = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.990864 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.990956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] use_syslog = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.991053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.991185 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 19:12:03.991292 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.991439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.991506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] auth.external = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.991678 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.991744 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.991943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.992075 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.992209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.992278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.992458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.992530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] auth.password = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.992692 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] auth.token = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.992801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.992939 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.993019 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.993163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.993242 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.993404 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.993492 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.993634 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.993722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.993877 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.993966 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.994111 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.994189 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.994338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.994413 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.994570 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] credential.caching = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.994650 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.994814 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.994893 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.995062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.995150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.995299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.995417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.995531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.995608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.995771 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.995858 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.996025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.996109 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.996276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.996354 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.996526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.996622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.996760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.996839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.997007 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.997087 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.997272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.997365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.997611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.997688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.997760 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.997838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.997983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.998072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.998219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.998298 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.998467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.998548 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] federation.caching = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.998715 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.998808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.998946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.999038 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.999179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.999261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.999415 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.999501 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.999659 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.999738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.999893 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:03.999972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.000165 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.000257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.000399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.000479 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.000636 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.000721 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.000875 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.001030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.001118 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.001203 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.001323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.001403 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] identity.caching = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.001554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.001632 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.001786 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.001865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.002020 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.002099 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.002250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.002328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.002487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.002568 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.002714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.002825 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.002942 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.003020 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.003177 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.003258 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.003413 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.003492 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.003641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.003718 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.003868 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.004002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.004117 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.004332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.004411 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.004588 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.004675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.004831 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.004920 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.005082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.005233 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.005348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.005543 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.005646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.005803 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.005914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.006040 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.006178 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.006299 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.006486 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.006602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.006700 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.006800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.006931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.007168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.007280 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.007437 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.007545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.007642 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.007829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.007940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.008102 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.008216 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.008348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.008467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.008566 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.008757 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.008863 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.009040 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.009151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.009253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.009448 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.009553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.009704 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.009814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.010412 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.010529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.010627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.010723 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.010828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.010828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.011017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.011114 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.011209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.011311 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.011429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.011528 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.011528 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.011707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.011801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.011906 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.012079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.012225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.012328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.012457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.012562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.012690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.012843 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.012949 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.013172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.013286 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.013409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.013514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.013664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.013777 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.013894 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.014122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.014234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.014368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.014479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.014609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.014744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.014847 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.015050 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.015153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.015328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.015412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.015498 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.015658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.015738 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.015941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.016050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.016207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.016290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.016450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.016579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.016691 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.016884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.016967 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.017162 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.017240 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.017387 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.017529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.017613 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.017796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.017876 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.018069 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.018154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.018302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.018404 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.018533 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.018751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.018831 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.019004 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.019090 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.019239 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.019393 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.019473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.019702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.019783 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.019941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.020102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.020209 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.020410 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] auth.external = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.020490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.020663 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.020752 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.020905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.021079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.021161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.021359 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.021437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.021593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.021674 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] auth.password = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.021831 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] resource.caching = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.021954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] auth.token = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.022076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.022271 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.022359 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.022541 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.022632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.022785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.022941 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.023020 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.023249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.023331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.023487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.023601 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.023720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.023918 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.024023 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.024197 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] role.caching = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.024287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.024442 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] role.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.024553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] credential.caching = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.024670 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.024913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.025050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.025155 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.025265 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.025399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.025598 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.025687 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.025873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.025958 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.026116 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.026196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.026358 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.026534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.026615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.026817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.026896 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.027060 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.027216 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.027304 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.027533 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.027611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.027776 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.027876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.028231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.028320 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.028463 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.028592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.028867 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.029036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.029130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.029358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.029499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.029711 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.029791 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.029975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.030062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] federation.caching = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.030238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.030318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.030493 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.030573 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.030731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.030820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.030963 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.031050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.031203 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.031283 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.031434 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.031523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.031694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.031773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.031938 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.032070 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.032208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.032302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.032445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.032526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.032678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.032764 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.032908 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.032997 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] identity.caching = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.033141 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] token.caching = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.033221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.033380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.033460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.033625 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.033714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.033862 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.033941 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.034125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.034203 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.034361 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.034450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.034594 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.034673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.034837 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.034916 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.035077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.035166 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.035313 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.035391 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.035548 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.035627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.035784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.035871 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.036030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.036112 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.036279 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.036359 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.036525 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.036618 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.036754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.036835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.036993 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.037074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.037283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.037371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.037524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.037603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.connection = **** {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.037766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.037856 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.037999 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.038080 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.038236 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.038318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.038475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.038555 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.038904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.039116 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.039208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.039288 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.039350 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.039409 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.039468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.039604 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.039695 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.039770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.039830 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.039959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.040109 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.040269 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.040410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.040541 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.040646 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.040762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.040855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.041003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.041146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.041249 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.041352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.041453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.041542 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.041671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.041784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.041887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80139) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 07 19:12:04.041992 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.042198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.042506 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.042869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.043178 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.043442 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.043714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.044026 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.044366 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.044621 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.044874 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.045130 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.045353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.045591 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.045826 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.046073 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.046336 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.046595 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.046959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.047242 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.047494 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.047735 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.047969 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.048230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.048496 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.048917 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.049182 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.049464 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.049834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.050172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.050451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.050876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.051138 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.051423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.051837 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.052188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.052433 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.052675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.052925 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.053189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.053432 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.053690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.053925 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.054173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.054414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.054652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.054893 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.055142 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.055387 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.055623 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.055869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.056138 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] resource.caching = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.056394 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.056648 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.056888 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.057147 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.057394 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.057725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.057869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.058109 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.058352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.058587 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] role.caching = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.059192 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] role.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.059434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.059686 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.060010 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.060268 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.060518 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.060766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.061046 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.061329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.061608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.061944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.062204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.062516 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.062853 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.063213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.063771 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.064119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.064382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.064639 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.064892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.065149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.065392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.065798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.066059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.066304 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.066542 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.066779 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.067014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.067270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.067529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.067780 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.068044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.068286 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.068523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.068760 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] token.caching = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.069001 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.069253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.069491 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.069749 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.070000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.070245 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.070489 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.070734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.070985 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.071227 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.071469 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.071616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.071704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.071849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.071933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.072087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.072211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.072449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.072697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.072998 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.073267 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.connection = **** {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.073518 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.073772 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.074012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.074306 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.074576 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.074850 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.075124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.075392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.075690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.075959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.076272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.076598 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.076851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.077091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.077340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.077574 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.077816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.078053 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 19:12:04.078227 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80138) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 07 19:12:04.082486 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.103431 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.103642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.103891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.110824 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.111040 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.113863 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.132250 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.132458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.162458 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.169576 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.213848 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.214093 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.219367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.219602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.281093 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.281512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.281512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.281861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.282025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.282178 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.284747 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.285107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.285701 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.287001 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.287479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.287764 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.308285 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.308519 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.308747 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.309339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.309438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.309588 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.309699 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.309805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.309955 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.310042 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.310203 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.310421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.310655 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.310869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.321278 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.321848 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.334365 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.334600 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.334723 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.334828 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.334908 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.335025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.335115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.335264 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.335401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.335483 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.335556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.335629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.335701 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80138) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.335865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80139) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 19:12:04.352739 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.352993 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 07 19:12:04.353114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.353267 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 07 19:12:04.353390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.353491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 07 19:12:04.353556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.353732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.353856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 07 19:12:04.353967 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.354069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.354253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.354388 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.354458 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.354537 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.354799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.365915 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.366196 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.366391 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.366491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.366705 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.366852 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.366958 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.367276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.373607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.374401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.376680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.376961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.377240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.377523 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.377649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.377802 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.377940 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.378042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.378130 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.378325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.378446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.378593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.378682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.378874 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.379000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.379157 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.379292 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.379432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.379572 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.379704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.379983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.380272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.402540 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.403401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.404030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.404333 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.404978 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.405264 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.409084 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.409340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.409646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.409986 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.410270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.410562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.416262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.416550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.417301 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.417593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.420707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.420996 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.421287 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.421590 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.421752 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.421856 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.422047 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.422135 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.422347 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.422435 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.422638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.422737 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.422930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.423282 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.423599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.423930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.438822 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.439104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.440995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.441334 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.443104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.445624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.446207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.446502 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.448796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.449103 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.451025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.453706 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.454285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.457148 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.457495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.457752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.458056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.458375 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.458611 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.458882 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.460386 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.460689 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.461006 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.461299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.461577 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.461866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.474854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.475120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.475461 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.475684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.478286 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.478642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.478915 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.479181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.482866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.484470 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.484707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.486745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.487464 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.487755 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.488312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.488447 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.488589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.488717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.488868 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.491376 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.491667 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.491956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.492283 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.492568 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.500340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.500626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.500906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.501188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.501460 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.501732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.502003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.504420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.504713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.504999 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.505284 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.505568 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.505846 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.506124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.518958 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.521942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.524537 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.524888 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.526461 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.527644 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.528268 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.528501 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.528853 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.530896 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.532677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.534267 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.535450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.535946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.536139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.537115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.540043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.540333 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.540623 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.540898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.542918 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.544598 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.547573 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.547894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.548240 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.548548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.549410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.549725 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.549991 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.557020 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80139) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.557179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.557405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.557529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.557754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.557877 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.558145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.558486 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.559172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.559554 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.559927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.560481 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.560819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.561138 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.561473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.561828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.562126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80139) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.565756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80138) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 19:12:04.566063 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.566329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.566616 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.566895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.567170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.567448 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.567720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.568138 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.568486 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.568792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.570820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.570820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.570820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80138) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 19:12:04.596018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5610c5c9b9c0 pid: 80139 (default app) Aug 07 19:12:04.599420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b5c9f-20d0-438f-978d-b34755a0febb None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:04.599700 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b5c9f-20d0-438f-978d-b34755a0febb None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:04.599973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b5c9f-20d0-438f-978d-b34755a0febb None None] PATH_INFO: `/v3` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:04.604339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Sun Aug 7 19:12:04 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:04.604584 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5610c5c9b9c0 pid: 80138 (default app) Aug 07 19:12:12.276532 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25857be-78ca-497f-a083-a94c2ed45e38 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:12.277083 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25857be-78ca-497f-a083-a94c2ed45e38 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:12.277083 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25857be-78ca-497f-a083-a94c2ed45e38 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:12.278308 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 2/2] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:12.286091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a71f46-e38c-44e2-8f82-ee806bba1e6b None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:12.286567 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a71f46-e38c-44e2-8f82-ee806bba1e6b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:12.286658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a71f46-e38c-44e2-8f82-ee806bba1e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:12.355785 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: INFO dbcounter [None req-67a71f46-e38c-44e2-8f82-ee806bba1e6b None None] Registered counter for database keystone Aug 07 19:12:12.375212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG oslo_db.sqlalchemy.engines [None req-67a71f46-e38c-44e2-8f82-ee806bba1e6b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80138) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 19:12:12.382703 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writer thread running {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 19:12:12.564868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG passlib.handlers.bcrypt [None req-67a71f46-e38c-44e2-8f82-ee806bba1e6b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80138) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 07 19:12:12.565164 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG passlib.handlers.bcrypt [None req-67a71f46-e38c-44e2-8f82-ee806bba1e6b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80138) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 07 19:12:12.619875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-67a71f46-e38c-44e2-8f82-ee806bba1e6b None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:12.621510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-67a71f46-e38c-44e2-8f82-ee806bba1e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:12.760703 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1/3] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 07 19:12:12.897986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25857be-78ca-497f-a083-a94c2ed45e38 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:12.967206 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: INFO dbcounter [None req-b25857be-78ca-497f-a083-a94c2ed45e38 None None] Registered counter for database keystone Aug 07 19:12:12.988719 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG oslo_db.sqlalchemy.engines [None req-b25857be-78ca-497f-a083-a94c2ed45e38 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80139) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 19:12:12.995886 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writer thread running {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 19:12:13.112246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25857be-78ca-497f-a083-a94c2ed45e38 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:13.117118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22353339-c910-4642-b1ca-215ff9d97192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:13.117754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22353339-c910-4642-b1ca-215ff9d97192 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:13.118039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22353339-c910-4642-b1ca-215ff9d97192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:13.118312 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22353339-c910-4642-b1ca-215ff9d97192 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:13.126784 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22353339-c910-4642-b1ca-215ff9d97192 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:13.178278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: WARNING keystone.common.rbac_enforcer.enforcer [None req-22353339-c910-4642-b1ca-215ff9d97192 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 07 19:12:13.178624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22353339-c910-4642-b1ca-215ff9d97192 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:13.188911 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 3/4] 10.222.0.17 () {58 vars in 1324 bytes} [Sun Aug 7 19:12:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 19:12:13.195183 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a71f46-e38c-44e2-8f82-ee806bba1e6b None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:13.210222 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a71f46-e38c-44e2-8f82-ee806bba1e6b None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:13.216012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdab385-6407-4cd5-9008-a101d026bc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:13.217172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdab385-6407-4cd5-9008-a101d026bc54 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:13.217453 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdab385-6407-4cd5-9008-a101d026bc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:13.217725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdab385-6407-4cd5-9008-a101d026bc54 admin admin] PATH_INFO: `/v3/projects/6c00a4064cc8477492c9b8b2e5412197` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:13.219339 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdab385-6407-4cd5-9008-a101d026bc54 admin admin] RBAC: Authorizing `identity:get_project(project_id=6c00a4064cc8477492c9b8b2e5412197)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:13.269395 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ebdab385-6407-4cd5-9008-a101d026bc54 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 07 19:12:13.269740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdab385-6407-4cd5-9008-a101d026bc54 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:13.274332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 2/5] 10.222.0.17 () {56 vars in 1207 bytes} [Sun Aug 7 19:12:13 2022] GET /identity/v3/projects/6c00a4064cc8477492c9b8b2e5412197 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:15.251736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea24c66-348f-4f13-aff8-83fe76bb1859 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:15.252334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea24c66-348f-4f13-aff8-83fe76bb1859 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:15.252423 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea24c66-348f-4f13-aff8-83fe76bb1859 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:15.256491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 4/6] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:15.262596 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca5473-6680-41ea-8c99-47750e4bb3dc None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:15.263073 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca5473-6680-41ea-8c99-47750e4bb3dc None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:15.263073 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca5473-6680-41ea-8c99-47750e4bb3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:15.286107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-abca5473-6680-41ea-8c99-47750e4bb3dc None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:15.287662 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-abca5473-6680-41ea-8c99-47750e4bb3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:15.296716 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 3/7] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 07 19:12:15.319393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea24c66-348f-4f13-aff8-83fe76bb1859 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:15.333561 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea24c66-348f-4f13-aff8-83fe76bb1859 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:15.338496 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0903460-85b5-4b6f-8500-ad6b29d1946d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:15.339113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0903460-85b5-4b6f-8500-ad6b29d1946d admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:15.339394 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0903460-85b5-4b6f-8500-ad6b29d1946d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:15.339667 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0903460-85b5-4b6f-8500-ad6b29d1946d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:15.348143 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0903460-85b5-4b6f-8500-ad6b29d1946d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:15.349944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0903460-85b5-4b6f-8500-ad6b29d1946d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:15.358718 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 5/8] 10.222.0.17 () {58 vars in 1324 bytes} [Sun Aug 7 19:12:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 19:12:15.364766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca5473-6680-41ea-8c99-47750e4bb3dc None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:15.374589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca5473-6680-41ea-8c99-47750e4bb3dc None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:15.380459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190267a-4a42-4af6-bc69-f670fece8ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:15.381105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190267a-4a42-4af6-bc69-f670fece8ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:15.381383 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190267a-4a42-4af6-bc69-f670fece8ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:15.381653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190267a-4a42-4af6-bc69-f670fece8ac2 admin admin] PATH_INFO: `/v3/users/39e61fe22f50447d975033301fd5b706` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:15.383129 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190267a-4a42-4af6-bc69-f670fece8ac2 admin admin] RBAC: Authorizing `identity:get_user(user_id=39e61fe22f50447d975033301fd5b706)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:15.385042 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190267a-4a42-4af6-bc69-f670fece8ac2 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:15.386614 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 4/9] 10.222.0.17 () {56 vars in 1198 bytes} [Sun Aug 7 19:12:15 2022] GET /identity/v3/users/39e61fe22f50447d975033301fd5b706 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:17.502867 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911b1c9-6231-4fef-b999-6bf29cdf2a9a None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:17.503416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911b1c9-6231-4fef-b999-6bf29cdf2a9a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:17.503416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911b1c9-6231-4fef-b999-6bf29cdf2a9a None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:17.504917 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 6/10] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:17.510632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202619ec-d481-42f6-b1b9-acbdd61cc464 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:17.511115 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202619ec-d481-42f6-b1b9-acbdd61cc464 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:17.511227 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202619ec-d481-42f6-b1b9-acbdd61cc464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:17.530102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-202619ec-d481-42f6-b1b9-acbdd61cc464 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:17.531368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-202619ec-d481-42f6-b1b9-acbdd61cc464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:17.538416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40a22c-72aa-49d0-814f-1fe5864f11a4 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:17.538702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40a22c-72aa-49d0-814f-1fe5864f11a4 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:17.538972 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40a22c-72aa-49d0-814f-1fe5864f11a4 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:17.540470 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 7/11] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:17.546439 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0af101-481a-4118-bbc5-a6edef3d68b3 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:17.546695 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0af101-481a-4118-bbc5-a6edef3d68b3 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:17.546976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0af101-481a-4118-bbc5-a6edef3d68b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:17.591791 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG passlib.handlers.bcrypt [None req-aa0af101-481a-4118-bbc5-a6edef3d68b3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80139) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 07 19:12:17.592252 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG passlib.handlers.bcrypt [None req-aa0af101-481a-4118-bbc5-a6edef3d68b3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80139) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 07 19:12:17.616284 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 5/12] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:17.637600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-aa0af101-481a-4118-bbc5-a6edef3d68b3 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:17.638433 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202619ec-d481-42f6-b1b9-acbdd61cc464 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:17.639290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-aa0af101-481a-4118-bbc5-a6edef3d68b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:17.665366 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:17.744223 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 8/13] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:12:17.756698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202619ec-d481-42f6-b1b9-acbdd61cc464 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:17.767045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0af101-481a-4118-bbc5-a6edef3d68b3 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:17.800731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5bd888-e507-4ce4-bc7b-ad5a8c7f4ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:17.801357 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bd888-e507-4ce4-bc7b-ad5a8c7f4ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:17.801640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bd888-e507-4ce4-bc7b-ad5a8c7f4ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:17.801914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bd888-e507-4ce4-bc7b-ad5a8c7f4ca0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:17.873436 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5bd888-e507-4ce4-bc7b-ad5a8c7f4ca0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:17.874943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5bd888-e507-4ce4-bc7b-ad5a8c7f4ca0 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:17.879970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0af101-481a-4118-bbc5-a6edef3d68b3 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:17.882542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:17.899295 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:17.918929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4643603-cdec-4c9b-99da-c9df888b5a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:17.920278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4643603-cdec-4c9b-99da-c9df888b5a8f None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:17.920633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4643603-cdec-4c9b-99da-c9df888b5a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:17.920910 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4643603-cdec-4c9b-99da-c9df888b5a8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:17.933425 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4643603-cdec-4c9b-99da-c9df888b5a8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:17.942762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-d4643603-cdec-4c9b-99da-c9df888b5a8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:17.942762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:17.942762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:17.942762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:17.942762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:17.942762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:17.942762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:17.942762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:17.942762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:17.942762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:17.942762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:17.942762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:17.942762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:17.942762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:17.942762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:17.942762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:17.943786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:17.943786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:17.943786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:17.943786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:17.943786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:17.943786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:17.943786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:17.943786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:17.943786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:17.943786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:17.943786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:17.943786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:17.943786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:17.943786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:17.943786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:17.943786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:17.944650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:17.944650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:17.944650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:17.944650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:17.944650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:17.944650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:17.944650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:17.944650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:17.944650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:17.944650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:17.944650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:17.944650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:17.944650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 9/14] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:12:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:17.951687 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4643603-cdec-4c9b-99da-c9df888b5a8f None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:17.981845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 6/15] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 19:12:17.987721 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5bd888-e507-4ce4-bc7b-ad5a8c7f4ca0 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:18.067464 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4643603-cdec-4c9b-99da-c9df888b5a8f None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:18.109854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5faae-7bc1-4c30-b387-77e31c706b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:18.110487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5faae-7bc1-4c30-b387-77e31c706b2b None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:18.110780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5faae-7bc1-4c30-b387-77e31c706b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:18.111024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5faae-7bc1-4c30-b387-77e31c706b2b None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:18.111729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5faae-7bc1-4c30-b387-77e31c706b2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:18.112038 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5faae-7bc1-4c30-b387-77e31c706b2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:18.113537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5faae-7bc1-4c30-b387-77e31c706b2b None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:18.126800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 10/16] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:18.133584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5faae-7bc1-4c30-b387-77e31c706b2b None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:18.162329 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:18.203501 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5bd888-e507-4ce4-bc7b-ad5a8c7f4ca0 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:18.227076 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:18.239626 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fd052-c08a-4b04-af5d-95b1b9a90b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:18.240295 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fd052-c08a-4b04-af5d-95b1b9a90b8e None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:18.240574 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fd052-c08a-4b04-af5d-95b1b9a90b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:18.240845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fd052-c08a-4b04-af5d-95b1b9a90b8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:18.243216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5faae-7bc1-4c30-b387-77e31c706b2b None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:18.252098 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7fd052-c08a-4b04-af5d-95b1b9a90b8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:18.255495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7fd052-c08a-4b04-af5d-95b1b9a90b8e None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:18.272589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-7b7fd052-c08a-4b04-af5d-95b1b9a90b8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:12:18.272589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:18.272589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:18.272589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:18.272589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:18.272589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:18.272589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:18.272589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:18.272589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:18.272589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:18.272589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:18.272589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:18.272589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:12:18.272589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:12:18.272589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:12:18.272589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:12:18.273549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:18.273549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:18.273549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:18.273549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:18.273549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:18.273549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:18.273549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:18.273549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:18.273549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:18.273549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:18.273549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:18.273549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:18.273549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:18.273549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:18.273549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:18.273549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:18.273549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:12:18.274529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:12:18.274529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:12:18.274529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:12:18.274529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 7/17] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:18.279766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fd052-c08a-4b04-af5d-95b1b9a90b8e None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:18.279898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d076f-48ec-483f-b451-7421ce0e7783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:18.280510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d076f-48ec-483f-b451-7421ce0e7783 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:18.280792 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d076f-48ec-483f-b451-7421ce0e7783 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:18.281178 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d076f-48ec-483f-b451-7421ce0e7783 None admin] PATH_INFO: `/v3/users/39e61fe22f50447d975033301fd5b706` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:18.282723 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d076f-48ec-483f-b451-7421ce0e7783 None admin] RBAC: Authorizing `identity:get_user(user_id=39e61fe22f50447d975033301fd5b706)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:18.284216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d076f-48ec-483f-b451-7421ce0e7783 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:18.285840 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 11/18] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:18 2022] GET /identity/v3/users/39e61fe22f50447d975033301fd5b706 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:18.291503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d076f-48ec-483f-b451-7421ce0e7783 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:18.381962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fd052-c08a-4b04-af5d-95b1b9a90b8e None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:18.388130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:18.395178 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d076f-48ec-483f-b451-7421ce0e7783 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:18.417303 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639bab6-5327-47e7-89e2-977948b6f76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:18.417924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639bab6-5327-47e7-89e2-977948b6f76a None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:18.418186 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639bab6-5327-47e7-89e2-977948b6f76a None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:18.418461 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639bab6-5327-47e7-89e2-977948b6f76a None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:18.419212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639bab6-5327-47e7-89e2-977948b6f76a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:18.419509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639bab6-5327-47e7-89e2-977948b6f76a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:18.420950 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639bab6-5327-47e7-89e2-977948b6f76a None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:18.430509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9eaac0-4415-442b-806f-00f65d15a7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:18.431163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9eaac0-4415-442b-806f-00f65d15a7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:18.431456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9eaac0-4415-442b-806f-00f65d15a7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:18.431724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9eaac0-4415-442b-806f-00f65d15a7d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:18.432887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9eaac0-4415-442b-806f-00f65d15a7d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:18.434756 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9eaac0-4415-442b-806f-00f65d15a7d7 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:18.436386 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 12/19] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:12:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:18.439096 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 8/20] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:18.442553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9eaac0-4415-442b-806f-00f65d15a7d7 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:18.559151 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9eaac0-4415-442b-806f-00f65d15a7d7 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:18.601594 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3b85b-ca5f-4fd4-a662-1873a4dd5a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:18.602220 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3b85b-ca5f-4fd4-a662-1873a4dd5a25 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:18.602502 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3b85b-ca5f-4fd4-a662-1873a4dd5a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:18.602775 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3b85b-ca5f-4fd4-a662-1873a4dd5a25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:18.603493 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3b85b-ca5f-4fd4-a662-1873a4dd5a25 None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.domain.id=default, user.id=39e61fe22f50447d975033301fd5b706) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:18.603800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3b85b-ca5f-4fd4-a662-1873a4dd5a25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:18.605293 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3b85b-ca5f-4fd4-a662-1873a4dd5a25 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:18.614671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 13/21] 10.222.0.17 () {56 vars in 1322 bytes} [Sun Aug 7 19:12:18 2022] GET /identity/v3/role_assignments?user.id=39e61fe22f50447d975033301fd5b706&scope.domain.id=default&role.id=4da42f08d107448fb877b42b61bc74ea => generated 233 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:20.938931 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890c699-6602-469a-8146-113e8f11f3e5 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:20.939467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890c699-6602-469a-8146-113e8f11f3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:20.939745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890c699-6602-469a-8146-113e8f11f3e5 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:20.942360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 9/22] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:20.948390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efb666-22a2-49fc-9921-77f1aba604c7 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:20.948925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efb666-22a2-49fc-9921-77f1aba604c7 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:20.949204 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53efb666-22a2-49fc-9921-77f1aba604c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:20.979707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-53efb666-22a2-49fc-9921-77f1aba604c7 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:20.981111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-53efb666-22a2-49fc-9921-77f1aba604c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:21.006458 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:21.033599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424b727-728f-416b-b1ef-5bebf0ed059e None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:21.034027 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424b727-728f-416b-b1ef-5bebf0ed059e None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:21.034441 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424b727-728f-416b-b1ef-5bebf0ed059e None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:21.036986 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 10/23] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:21.043258 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f6aff-c5f5-4589-b8ab-b8377256b402 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:21.043920 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f6aff-c5f5-4589-b8ab-b8377256b402 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:21.044386 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f6aff-c5f5-4589-b8ab-b8377256b402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:21.098003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-385f6aff-c5f5-4589-b8ab-b8377256b402 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:21.099231 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-385f6aff-c5f5-4589-b8ab-b8377256b402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:21.140795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:21.144558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 14/24] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:21.146186 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54287ec6-0701-4be9-a4b6-5d1e2fef4751 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:21.146478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54287ec6-0701-4be9-a4b6-5d1e2fef4751 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:21.146754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54287ec6-0701-4be9-a4b6-5d1e2fef4751 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:21.148640 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 15/25] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:21.159266 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf8f75-420b-4a66-b6b9-fa3ada2a3a2a None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:21.159540 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf8f75-420b-4a66-b6b9-fa3ada2a3a2a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:21.159820 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf8f75-420b-4a66-b6b9-fa3ada2a3a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:21.199444 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-bacf8f75-420b-4a66-b6b9-fa3ada2a3a2a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:21.200660 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-bacf8f75-420b-4a66-b6b9-fa3ada2a3a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:21.236485 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 11/26] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:21.240002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f6aff-c5f5-4589-b8ab-b8377256b402 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:21.458825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 16/27] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:12:21.465060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f6aff-c5f5-4589-b8ab-b8377256b402 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:21.468840 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf8f75-420b-4a66-b6b9-fa3ada2a3a2a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:21.510463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877122a-f4cd-48d5-aa6b-299030c9633b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:21.510463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877122a-f4cd-48d5-aa6b-299030c9633b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:21.510463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877122a-f4cd-48d5-aa6b-299030c9633b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:21.510463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877122a-f4cd-48d5-aa6b-299030c9633b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:21.529061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877122a-f4cd-48d5-aa6b-299030c9633b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:21.536214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-6877122a-f4cd-48d5-aa6b-299030c9633b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:21.536214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:21.536214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:21.536214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:21.536214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:21.536214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:21.536214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:21.536214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:21.536214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:21.536214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:21.536214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:21.536214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:21.536214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:21.536214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:21.536214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:21.536214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:21.537072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:21.537072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:21.537072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:21.537072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:21.537072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:21.537072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:21.537072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:21.537072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:21.537072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:21.537072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:21.537072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:21.537072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:21.537072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:21.537072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:21.537072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:21.537072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:21.537814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:21.537814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:21.537814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:21.537814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:21.537814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:21.537814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:21.537814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:21.537814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:21.537814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:21.537814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:21.537814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:21.537814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:12:21.540133 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 12/28] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:12:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:21.541168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877122a-f4cd-48d5-aa6b-299030c9633b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:21.572949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:21.609590 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:21.629914 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf8f75-420b-4a66-b6b9-fa3ada2a3a2a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:21.670778 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b8f74-7fb2-4b0c-8cf2-3413169b7eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:21.672720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b8f74-7fb2-4b0c-8cf2-3413169b7eeb None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:21.674094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b8f74-7fb2-4b0c-8cf2-3413169b7eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:21.674597 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b8f74-7fb2-4b0c-8cf2-3413169b7eeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:21.698229 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877122a-f4cd-48d5-aa6b-299030c9633b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:21.703597 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b8f74-7fb2-4b0c-8cf2-3413169b7eeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:21.705132 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b8f74-7fb2-4b0c-8cf2-3413169b7eeb None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:21.726531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-4c7b8f74-7fb2-4b0c-8cf2-3413169b7eeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:12:21.726531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:21.726531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:21.726531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:21.726531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:21.726531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:21.726531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:21.726531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:21.726531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:21.726531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:21.726531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:21.726531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:21.726531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:12:21.726531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:12:21.726531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:12:21.726531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:12:21.727414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:21.727414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:21.727414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:21.727414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:21.727414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:21.727414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:21.727414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:21.727414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:21.727414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:21.727414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:21.727414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:21.727414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:21.727414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:21.727414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:21.727414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:21.727414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:21.727414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:12:21.728264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:12:21.728264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:12:21.728264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:21.728264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 17/29] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:21.732551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b8f74-7fb2-4b0c-8cf2-3413169b7eeb None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:21.748379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670a8d6-9228-4c03-a0fb-a266f777c973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:21.749424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670a8d6-9228-4c03-a0fb-a266f777c973 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:21.749891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670a8d6-9228-4c03-a0fb-a266f777c973 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:21.750414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670a8d6-9228-4c03-a0fb-a266f777c973 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:21.751391 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670a8d6-9228-4c03-a0fb-a266f777c973 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:21.753044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4670a8d6-9228-4c03-a0fb-a266f777c973 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:21.858436 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b8f74-7fb2-4b0c-8cf2-3413169b7eeb None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:21.898942 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 13/30] 10.222.0.17 () {60 vars in 1131 bytes} [Sun Aug 7 19:12:21 2022] POST /identity/v3/roles => generated 225 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:12:21.900796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66902b2c-8ef1-4af5-ac1e-4a8423cc4874 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:21.900969 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66902b2c-8ef1-4af5-ac1e-4a8423cc4874 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:21.901254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66902b2c-8ef1-4af5-ac1e-4a8423cc4874 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:21.902551 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 14/31] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:21.903526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66902b2c-8ef1-4af5-ac1e-4a8423cc4874 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:21.916389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd9549-9cf4-467d-8188-c7b19f1e4a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:21.917074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd9549-9cf4-467d-8188-c7b19f1e4a46 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:21.917325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd9549-9cf4-467d-8188-c7b19f1e4a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:21.917607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd9549-9cf4-467d-8188-c7b19f1e4a46 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:21.918441 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd9549-9cf4-467d-8188-c7b19f1e4a46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:21.918758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd9549-9cf4-467d-8188-c7b19f1e4a46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:21.920285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd9549-9cf4-467d-8188-c7b19f1e4a46 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:21.927319 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:21.936488 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 18/32] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:21.938098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489018a4-9335-4a30-9548-b125a449beab None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:21.938391 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489018a4-9335-4a30-9548-b125a449beab None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:21.938671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489018a4-9335-4a30-9548-b125a449beab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:21.961078 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-489018a4-9335-4a30-9548-b125a449beab None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:21.961987 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-489018a4-9335-4a30-9548-b125a449beab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:22.033998 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:22.079810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66902b2c-8ef1-4af5-ac1e-4a8423cc4874 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:22.081524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 19/33] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:22.082553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489018a4-9335-4a30-9548-b125a449beab None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:22.135937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159805c-3174-4df8-bb9d-f2a41cbb1506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:22.136635 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159805c-3174-4df8-bb9d-f2a41cbb1506 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:22.136919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159805c-3174-4df8-bb9d-f2a41cbb1506 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:22.137194 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159805c-3174-4df8-bb9d-f2a41cbb1506 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:22.138014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159805c-3174-4df8-bb9d-f2a41cbb1506 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:22.138330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159805c-3174-4df8-bb9d-f2a41cbb1506 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:22.140076 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159805c-3174-4df8-bb9d-f2a41cbb1506 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:22.154170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 15/34] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:22.155294 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a43f29-a00e-4d37-80cd-dfbbb336687c None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:22.155739 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a43f29-a00e-4d37-80cd-dfbbb336687c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:22.156177 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a43f29-a00e-4d37-80cd-dfbbb336687c None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:22.157822 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 16/35] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:22.158524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a43f29-a00e-4d37-80cd-dfbbb336687c None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:22.228445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489018a4-9335-4a30-9548-b125a449beab None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:22.260797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:22.267725 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3707ed-6d0e-4c01-87f3-abfaa2b6ca21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:22.268515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3707ed-6d0e-4c01-87f3-abfaa2b6ca21 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:22.268796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3707ed-6d0e-4c01-87f3-abfaa2b6ca21 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:22.269070 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3707ed-6d0e-4c01-87f3-abfaa2b6ca21 None admin] PATH_INFO: `/v3/users/39e61fe22f50447d975033301fd5b706` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:22.270344 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3707ed-6d0e-4c01-87f3-abfaa2b6ca21 None admin] RBAC: Authorizing `identity:get_user(user_id=39e61fe22f50447d975033301fd5b706)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:22.271981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3707ed-6d0e-4c01-87f3-abfaa2b6ca21 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:22.273754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 20/36] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:22 2022] GET /identity/v3/users/39e61fe22f50447d975033301fd5b706 => generated 247 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:22.274512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3707ed-6d0e-4c01-87f3-abfaa2b6ca21 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:22.293459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a43f29-a00e-4d37-80cd-dfbbb336687c None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:22.336776 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320a11d-821b-4c41-9807-ae986e2b601e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:22.337488 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320a11d-821b-4c41-9807-ae986e2b601e None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:22.337831 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320a11d-821b-4c41-9807-ae986e2b601e None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:22.338186 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320a11d-821b-4c41-9807-ae986e2b601e None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:22.338927 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320a11d-821b-4c41-9807-ae986e2b601e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:22.340456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320a11d-821b-4c41-9807-ae986e2b601e None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:22.355754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 17/37] 10.222.0.17 () {60 vars in 1131 bytes} [Sun Aug 7 19:12:22 2022] POST /identity/v3/roles => generated 219 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:12:22.356843 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b88c6-5f32-4b5e-84ae-a82563093ab1 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:22.357130 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b88c6-5f32-4b5e-84ae-a82563093ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:22.357405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b88c6-5f32-4b5e-84ae-a82563093ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:22.365895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:22.380424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-863b88c6-5f32-4b5e-84ae-a82563093ab1 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:22.382237 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-863b88c6-5f32-4b5e-84ae-a82563093ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:22.435473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3707ed-6d0e-4c01-87f3-abfaa2b6ca21 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:22.475835 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df420944-66ae-42e5-b8bf-73736b4f1036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:22.476554 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df420944-66ae-42e5-b8bf-73736b4f1036 None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:22.476942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df420944-66ae-42e5-b8bf-73736b4f1036 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:22.477173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df420944-66ae-42e5-b8bf-73736b4f1036 None admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:22.477843 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df420944-66ae-42e5-b8bf-73736b4f1036 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:22.479506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df420944-66ae-42e5-b8bf-73736b4f1036 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:22.492591 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 18/38] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:22.493973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b88c6-5f32-4b5e-84ae-a82563093ab1 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:22.613387 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b88c6-5f32-4b5e-84ae-a82563093ab1 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:22.638733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 21/39] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Aug 7 19:12:22 2022] POST /identity/v3/projects => generated 301 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:12:22.639557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df420944-66ae-42e5-b8bf-73736b4f1036 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:22.653649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ea8fb-fef6-4d0d-a6e2-6dadb2a66b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:22.654323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ea8fb-fef6-4d0d-a6e2-6dadb2a66b35 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:22.654597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ea8fb-fef6-4d0d-a6e2-6dadb2a66b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:22.654873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ea8fb-fef6-4d0d-a6e2-6dadb2a66b35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:22.656247 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ea8fb-fef6-4d0d-a6e2-6dadb2a66b35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:22.657802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ea8fb-fef6-4d0d-a6e2-6dadb2a66b35 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:22.667673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 19/40] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:12:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:22.667673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aad182-40ee-4c65-be5d-b6ad1c9d4481 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:22.667673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aad182-40ee-4c65-be5d-b6ad1c9d4481 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:22.667673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aad182-40ee-4c65-be5d-b6ad1c9d4481 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:22.667673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 20/41] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:22.669496 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aad182-40ee-4c65-be5d-b6ad1c9d4481 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:22.674176 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:22.739279 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:22.811206 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df420944-66ae-42e5-b8bf-73736b4f1036 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:22.822751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aad182-40ee-4c65-be5d-b6ad1c9d4481 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:22.868285 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f539b-a413-447f-ad49-a34d330ca63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:22.868876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f539b-a413-447f-ad49-a34d330ca63c None admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:22.869162 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f539b-a413-447f-ad49-a34d330ca63c None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:22.869438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f539b-a413-447f-ad49-a34d330ca63c None admin] PATH_INFO: `/v3/domains/default/users/39e61fe22f50447d975033301fd5b706/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:22.871466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f539b-a413-447f-ad49-a34d330ca63c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=39e61fe22f50447d975033301fd5b706, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:22.873081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464f539b-a413-447f-ad49-a34d330ca63c None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:22.878559 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0f1e8-9728-4276-bb6e-c18f3e901670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:22.880205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0f1e8-9728-4276-bb6e-c18f3e901670 None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:22.880781 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0f1e8-9728-4276-bb6e-c18f3e901670 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:22.881288 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0f1e8-9728-4276-bb6e-c18f3e901670 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:22.881977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0f1e8-9728-4276-bb6e-c18f3e901670 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:22.883702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0f1e8-9728-4276-bb6e-c18f3e901670 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:22.891424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 21/42] 10.222.0.17 () {58 vars in 1361 bytes} [Sun Aug 7 19:12:22 2022] PUT /identity/v3/domains/default/users/39e61fe22f50447d975033301fd5b706/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:12:22.892771 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7390e25-8363-44a9-a3fb-f9cbb603e1a1 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:22.893092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7390e25-8363-44a9-a3fb-f9cbb603e1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:22.893328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7390e25-8363-44a9-a3fb-f9cbb603e1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:22.907337 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 22/43] 10.222.0.17 () {60 vars in 1131 bytes} [Sun Aug 7 19:12:22 2022] POST /identity/v3/roles => generated 223 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:12:22.913800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-b7390e25-8363-44a9-a3fb-f9cbb603e1a1 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:22.915112 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b7390e25-8363-44a9-a3fb-f9cbb603e1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:23.016045 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:23.033524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 22/44] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:23.059591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0f1e8-9728-4276-bb6e-c18f3e901670 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:23.189038 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0f1e8-9728-4276-bb6e-c18f3e901670 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:23.201240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:23.233340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7fa95-98b4-4f0b-8966-1789d5edc5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:23.234421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7fa95-98b4-4f0b-8966-1789d5edc5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:23.234910 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7fa95-98b4-4f0b-8966-1789d5edc5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:23.235392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7fa95-98b4-4f0b-8966-1789d5edc5f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:23.237626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7fa95-98b4-4f0b-8966-1789d5edc5f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:23.240160 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7fa95-98b4-4f0b-8966-1789d5edc5f5 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:23.243106 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 23/45] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:12:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:23.253278 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7390e25-8363-44a9-a3fb-f9cbb603e1a1 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:23.354187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7390e25-8363-44a9-a3fb-f9cbb603e1a1 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:23.389125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b86a5-50cc-4606-b7ee-7ccd2023c5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:23.389739 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b86a5-50cc-4606-b7ee-7ccd2023c5ce None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:23.390021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b86a5-50cc-4606-b7ee-7ccd2023c5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:23.390287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b86a5-50cc-4606-b7ee-7ccd2023c5ce None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:23.390932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b86a5-50cc-4606-b7ee-7ccd2023c5ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:23.392438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7b86a5-50cc-4606-b7ee-7ccd2023c5ce None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:23.422963 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 23/46] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Aug 7 19:12:23 2022] POST /identity/v3/projects => generated 312 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:12:24.925688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525e1ec-f0d1-4fda-a339-50b5d675e1ad None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:24.926265 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525e1ec-f0d1-4fda-a339-50b5d675e1ad None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:24.926265 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525e1ec-f0d1-4fda-a339-50b5d675e1ad None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:24.927570 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 24/47] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:24.933209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eca1fe-1962-4e8f-9afe-51907bad4348 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:24.933988 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eca1fe-1962-4e8f-9afe-51907bad4348 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:24.934213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eca1fe-1962-4e8f-9afe-51907bad4348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:24.953668 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-18eca1fe-1962-4e8f-9afe-51907bad4348 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:24.954881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-18eca1fe-1962-4e8f-9afe-51907bad4348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:24.997534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:25.027674 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 24/48] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:25.050625 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525e1ec-f0d1-4fda-a339-50b5d675e1ad None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:25.159151 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525e1ec-f0d1-4fda-a339-50b5d675e1ad None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:25.193465 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e80aa-013d-4abf-b700-2a77eb04ca80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:25.194019 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e80aa-013d-4abf-b700-2a77eb04ca80 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:25.194300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e80aa-013d-4abf-b700-2a77eb04ca80 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:25.194570 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e80aa-013d-4abf-b700-2a77eb04ca80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:25.205115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e80aa-013d-4abf-b700-2a77eb04ca80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:25.209825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-393e80aa-013d-4abf-b700-2a77eb04ca80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:25.209825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:25.209825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:25.209825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:25.209825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:25.209825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:25.209825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:25.209825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:25.209825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:25.209825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:25.209825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:25.209825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:25.209825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:25.209825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:25.209825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:25.209825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:25.210705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:25.210705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:25.210705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:25.210705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:25.210705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:25.210705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:25.210705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:25.210705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:25.210705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:25.210705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:25.210705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:25.210705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:25.210705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:25.210705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:25.210705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:25.210705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:25.211539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:25.211539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:25.211539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:25.211539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:25.211539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:25.211539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:25.211539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:25.211539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:25.211539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:25.211539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:25.211539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:25.211539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:25.211539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 25/49] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:12:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:25.217243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eca1fe-1962-4e8f-9afe-51907bad4348 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:25.322451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eca1fe-1962-4e8f-9afe-51907bad4348 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:25.360330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd0d31-e4fd-4d22-acaa-10014ae3c3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:25.360949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd0d31-e4fd-4d22-acaa-10014ae3c3cc None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:25.361201 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd0d31-e4fd-4d22-acaa-10014ae3c3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:25.361509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd0d31-e4fd-4d22-acaa-10014ae3c3cc None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:25.362194 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd0d31-e4fd-4d22-acaa-10014ae3c3cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:25.362494 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd0d31-e4fd-4d22-acaa-10014ae3c3cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:25.365260 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd0d31-e4fd-4d22-acaa-10014ae3c3cc None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:25.378489 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 25/50] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:25.384245 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e80aa-013d-4abf-b700-2a77eb04ca80 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:25.430887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:25.448114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3846be-8374-4736-9740-892236ded9e2 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:25.448292 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3846be-8374-4736-9740-892236ded9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:25.448601 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3846be-8374-4736-9740-892236ded9e2 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:25.449936 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 26/51] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:12:25.455287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc672799-3e95-41a5-9815-c97a03df9e87 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:25.455574 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc672799-3e95-41a5-9815-c97a03df9e87 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:25.455851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc672799-3e95-41a5-9815-c97a03df9e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:25.474279 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-fc672799-3e95-41a5-9815-c97a03df9e87 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:25.475515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-fc672799-3e95-41a5-9815-c97a03df9e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:25.495771 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e80aa-013d-4abf-b700-2a77eb04ca80 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:25.505806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:25.530889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cde48a-7b53-4715-88bd-71043fcf3013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:25.531525 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cde48a-7b53-4715-88bd-71043fcf3013 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:25.531805 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cde48a-7b53-4715-88bd-71043fcf3013 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:25.532102 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cde48a-7b53-4715-88bd-71043fcf3013 None admin] PATH_INFO: `/v3/users/39e61fe22f50447d975033301fd5b706` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:25.533226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cde48a-7b53-4715-88bd-71043fcf3013 None admin] RBAC: Authorizing `identity:get_user(user_id=39e61fe22f50447d975033301fd5b706)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:25.534690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cde48a-7b53-4715-88bd-71043fcf3013 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:25.536326 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 26/52] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:25 2022] GET /identity/v3/users/39e61fe22f50447d975033301fd5b706 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:25.541964 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cde48a-7b53-4715-88bd-71043fcf3013 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:25.546450 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 27/53] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:12:25.569498 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc672799-3e95-41a5-9815-c97a03df9e87 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:25.647860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cde48a-7b53-4715-88bd-71043fcf3013 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:25.655129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:25.666756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc672799-3e95-41a5-9815-c97a03df9e87 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:25.686659 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f6e9a-42b4-4088-833e-8e54e6b2bf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:25.687309 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f6e9a-42b4-4088-833e-8e54e6b2bf40 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:25.687599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f6e9a-42b4-4088-833e-8e54e6b2bf40 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:25.687875 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f6e9a-42b4-4088-833e-8e54e6b2bf40 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:25.689111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f6e9a-42b4-4088-833e-8e54e6b2bf40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:25.690583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f6e9a-42b4-4088-833e-8e54e6b2bf40 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:25.692189 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 27/54] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:12:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:25.698294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f6e9a-42b4-4088-833e-8e54e6b2bf40 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:25.703336 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f683d7-35c1-4ed4-83c3-ab10972768a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:25.704007 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f683d7-35c1-4ed4-83c3-ab10972768a1 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:25.704317 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f683d7-35c1-4ed4-83c3-ab10972768a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:25.704592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f683d7-35c1-4ed4-83c3-ab10972768a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:25.705710 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f683d7-35c1-4ed4-83c3-ab10972768a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:25.707198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f683d7-35c1-4ed4-83c3-ab10972768a1 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:25.708816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 28/55] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:12:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:25.714799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f683d7-35c1-4ed4-83c3-ab10972768a1 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:25.750552 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:25.801269 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f6e9a-42b4-4088-833e-8e54e6b2bf40 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:25.818272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f683d7-35c1-4ed4-83c3-ab10972768a1 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:25.835580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e0bf7-0556-4d33-8629-d93dee2ac559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:25.836228 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e0bf7-0556-4d33-8629-d93dee2ac559 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:25.836515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e0bf7-0556-4d33-8629-d93dee2ac559 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:25.836794 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e0bf7-0556-4d33-8629-d93dee2ac559 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:25.837520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e0bf7-0556-4d33-8629-d93dee2ac559 None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.domain.id=default, user.id=39e61fe22f50447d975033301fd5b706) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:25.837856 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e0bf7-0556-4d33-8629-d93dee2ac559 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:25.839303 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2e0bf7-0556-4d33-8629-d93dee2ac559 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:25.845208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 28/56] 10.222.0.17 () {56 vars in 1322 bytes} [Sun Aug 7 19:12:25 2022] GET /identity/v3/role_assignments?user.id=39e61fe22f50447d975033301fd5b706&scope.domain.id=default&role.id=4da42f08d107448fb877b42b61bc74ea => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:25.854563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee80473-b78e-44c3-9dec-74444f08cbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:25.855198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee80473-b78e-44c3-9dec-74444f08cbb1 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:25.855477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee80473-b78e-44c3-9dec-74444f08cbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:25.855745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee80473-b78e-44c3-9dec-74444f08cbb1 None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:25.856489 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee80473-b78e-44c3-9dec-74444f08cbb1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:25.858045 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee80473-b78e-44c3-9dec-74444f08cbb1 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:25.896177 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 29/57] 10.222.0.17 () {60 vars in 1140 bytes} [Sun Aug 7 19:12:25 2022] POST /identity/v3/projects => generated 298 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:12:27.879296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38800350-fdc7-467a-9ece-786e3ba5b482 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:27.880220 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38800350-fdc7-467a-9ece-786e3ba5b482 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:27.880220 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38800350-fdc7-467a-9ece-786e3ba5b482 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:27.882350 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 29/58] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:27.891354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd73fc0-9ba2-4216-a04e-5d7541d4d8a9 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:27.891788 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd73fc0-9ba2-4216-a04e-5d7541d4d8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:27.892017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd73fc0-9ba2-4216-a04e-5d7541d4d8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:27.915187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-8fd73fc0-9ba2-4216-a04e-5d7541d4d8a9 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:27.916433 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8fd73fc0-9ba2-4216-a04e-5d7541d4d8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:27.989366 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 30/59] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:28.013774 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38800350-fdc7-467a-9ece-786e3ba5b482 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:28.060312 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:28.125498 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38800350-fdc7-467a-9ece-786e3ba5b482 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:28.161843 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2996b65-d31c-4b4e-96cd-f80dae959d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:28.162369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2996b65-d31c-4b4e-96cd-f80dae959d25 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:28.162653 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2996b65-d31c-4b4e-96cd-f80dae959d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:28.162928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2996b65-d31c-4b4e-96cd-f80dae959d25 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:28.164146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2996b65-d31c-4b4e-96cd-f80dae959d25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:28.165604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2996b65-d31c-4b4e-96cd-f80dae959d25 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:28.167205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 30/60] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:12:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:28.173311 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd73fc0-9ba2-4216-a04e-5d7541d4d8a9 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:28.193193 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:28.281595 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd73fc0-9ba2-4216-a04e-5d7541d4d8a9 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:28.317393 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937e410-a9f3-402f-9eec-c1a73bfd82ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:28.318008 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937e410-a9f3-402f-9eec-c1a73bfd82ca None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:28.318288 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937e410-a9f3-402f-9eec-c1a73bfd82ca None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:28.318570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937e410-a9f3-402f-9eec-c1a73bfd82ca None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:28.319209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937e410-a9f3-402f-9eec-c1a73bfd82ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:28.320709 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e937e410-a9f3-402f-9eec-c1a73bfd82ca None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:28.348554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 31/61] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:12:28 2022] POST /identity/v3/users => generated 275 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:12:31.574036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbaa11-5aa7-413b-beb9-c52a16339706 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:31.575048 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbaa11-5aa7-413b-beb9-c52a16339706 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:31.575562 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbaa11-5aa7-413b-beb9-c52a16339706 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:31.577168 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 31/62] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:31.582766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0125f79-912c-4edb-be3d-600c91559b27 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:31.583220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0125f79-912c-4edb-be3d-600c91559b27 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:31.583290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0125f79-912c-4edb-be3d-600c91559b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:31.597923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b631e3f-2a27-4386-9ebc-0481aa18acae None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:31.597923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b631e3f-2a27-4386-9ebc-0481aa18acae None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:31.597923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b631e3f-2a27-4386-9ebc-0481aa18acae None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:31.597923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 32/63] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:31.603764 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de51dc-c2f6-44bb-ac8d-52dd450ebbbc None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:31.604382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de51dc-c2f6-44bb-ac8d-52dd450ebbbc None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:31.604869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de51dc-c2f6-44bb-ac8d-52dd450ebbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:31.617530 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-c0125f79-912c-4edb-be3d-600c91559b27 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:31.618325 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c0125f79-912c-4edb-be3d-600c91559b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:31.628979 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-b0de51dc-c2f6-44bb-ac8d-52dd450ebbbc None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:31.630798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b0de51dc-c2f6-44bb-ac8d-52dd450ebbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:31.739460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 32/64] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:31.746077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 33/65] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:12:31.762596 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0125f79-912c-4edb-be3d-600c91559b27 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:31.764862 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:31.780542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de51dc-c2f6-44bb-ac8d-52dd450ebbbc None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:31.842698 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:31.880168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0125f79-912c-4edb-be3d-600c91559b27 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:31.919457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de51dc-c2f6-44bb-ac8d-52dd450ebbbc None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:31.920298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c004034-c379-4124-859f-239b0997d5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:31.921092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c004034-c379-4124-859f-239b0997d5eb None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:31.921446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c004034-c379-4124-859f-239b0997d5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:31.921781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c004034-c379-4124-859f-239b0997d5eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:31.932953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c004034-c379-4124-859f-239b0997d5eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:31.938515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-0c004034-c379-4124-859f-239b0997d5eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:31.938515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:31.938515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:31.938515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:31.938515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:31.938515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:31.938515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:31.938515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:31.938515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:31.938515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:31.938515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:31.938515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:31.938515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:31.938515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:31.938515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:31.938515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:31.939562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:31.939562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:31.939562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:31.939562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:31.939562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:31.939562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:31.939562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:31.939562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:31.939562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:31.939562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:31.939562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:31.939562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:31.939562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:31.939562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:31.939562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:31.939562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:31.940414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:31.940414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:31.940414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:31.940414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:31.940414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:31.940414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:31.940414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:31.940414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:31.940414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:31.940414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:31.940414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:31.940414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:12:31.940414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 33/66] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:12:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:31.941018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae79f0-ee50-45c2-a254-8e3742d38928 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:31.941428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae79f0-ee50-45c2-a254-8e3742d38928 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:31.941428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae79f0-ee50-45c2-a254-8e3742d38928 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:31.943273 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 34/67] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:31.947903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709412f-0bf9-4dfa-80ca-9849fd79e94e None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:31.947903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709412f-0bf9-4dfa-80ca-9849fd79e94e None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:31.947903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709412f-0bf9-4dfa-80ca-9849fd79e94e None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:31.947903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 35/68] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:31.947903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709412f-0bf9-4dfa-80ca-9849fd79e94e None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:31.960388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196aa5e2-fee5-4433-8d9a-41b6ae49be11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:31.961044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196aa5e2-fee5-4433-8d9a-41b6ae49be11 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:31.961325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196aa5e2-fee5-4433-8d9a-41b6ae49be11 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:31.961592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196aa5e2-fee5-4433-8d9a-41b6ae49be11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:31.973555 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196aa5e2-fee5-4433-8d9a-41b6ae49be11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:31.978746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-196aa5e2-fee5-4433-8d9a-41b6ae49be11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:31.978746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:31.978746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:31.978746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:31.978746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:31.978746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:31.978746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:31.978746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:31.978746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:31.978746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:31.978746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:31.978746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:31.978746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:31.978746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:31.978746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:31.978746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:31.979600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:31.979600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:31.979600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:31.979600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:31.979600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:31.979600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:31.979600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:31.979600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:31.979600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:31.979600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:31.979600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:31.979600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:31.979600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:31.979600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:31.979600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:31.979600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:31.980396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:31.980396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:31.980396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:31.980396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:31.980396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:31.980396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:31.980396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:31.980396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:31.980396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:31.980396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:31.980396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:31.980396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:31.980396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 34/69] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:12:31.981050 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b55ed-6b25-4634-a6e8-317b07918c4f None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:31.981825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b55ed-6b25-4634-a6e8-317b07918c4f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:31.981825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b55ed-6b25-4634-a6e8-317b07918c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:32.000596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-628b55ed-6b25-4634-a6e8-317b07918c4f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:32.001757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-628b55ed-6b25-4634-a6e8-317b07918c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:32.019613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:32.063215 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709412f-0bf9-4dfa-80ca-9849fd79e94e None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:32.084135 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 35/70] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:32.085491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722ba26-ac84-430a-ab91-e809323558ed None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:32.085787 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722ba26-ac84-430a-ab91-e809323558ed None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:32.086068 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722ba26-ac84-430a-ab91-e809323558ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:32.108362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-e722ba26-ac84-430a-ab91-e809323558ed None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:32.109512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e722ba26-ac84-430a-ab91-e809323558ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:32.110545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e8d94-113e-46ac-8ccf-aaf6549be9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:32.111170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e8d94-113e-46ac-8ccf-aaf6549be9ba None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:32.111447 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e8d94-113e-46ac-8ccf-aaf6549be9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:32.111885 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e8d94-113e-46ac-8ccf-aaf6549be9ba None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:32.113463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e8d94-113e-46ac-8ccf-aaf6549be9ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:32.113779 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e8d94-113e-46ac-8ccf-aaf6549be9ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:32.115198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e8d94-113e-46ac-8ccf-aaf6549be9ba None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:32.124136 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 36/71] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:32.125198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e8d94-113e-46ac-8ccf-aaf6549be9ba None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:32.136591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:32.188805 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 36/72] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:32.188805 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722ba26-ac84-430a-ab91-e809323558ed None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:32.231184 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e8d94-113e-46ac-8ccf-aaf6549be9ba None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:32.256406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:32.269230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73479f91-757e-4682-9f51-6d0177f0f404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:32.269965 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73479f91-757e-4682-9f51-6d0177f0f404 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:32.270342 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73479f91-757e-4682-9f51-6d0177f0f404 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:32.270669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73479f91-757e-4682-9f51-6d0177f0f404 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:32.271437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73479f91-757e-4682-9f51-6d0177f0f404 None admin] RBAC: Adding query filter params (name=member) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:32.271729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73479f91-757e-4682-9f51-6d0177f0f404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:32.273990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73479f91-757e-4682-9f51-6d0177f0f404 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:32.282932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 37/73] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:12:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:32.283767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73479f91-757e-4682-9f51-6d0177f0f404 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:32.301962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722ba26-ac84-430a-ab91-e809323558ed None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:32.340267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4d994-b4fa-4841-8fac-43e449aa8ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:32.340969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4d994-b4fa-4841-8fac-43e449aa8ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:32.341255 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4d994-b4fa-4841-8fac-43e449aa8ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:32.341528 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4d994-b4fa-4841-8fac-43e449aa8ed8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:32.352408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4d994-b4fa-4841-8fac-43e449aa8ed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:32.357042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-4af4d994-b4fa-4841-8fac-43e449aa8ed8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:12:32.357042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:32.357042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:32.357042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:32.357042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:32.357042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:32.357042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:32.357042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:32.357042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:32.357042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:32.357042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:32.357042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:32.357042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:32.357042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:32.357042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:32.357042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:32.357901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:32.357901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:32.357901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:32.357901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:32.357901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:32.357901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:32.357901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:32.357901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:32.357901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:32.357901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:32.357901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:32.357901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:32.357901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:32.357901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:32.357901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:32.357901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:32.358707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:32.358707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:32.358707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:32.358707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:32.358707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:32.358707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:32.358707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:32.358707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:32.358707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:32.358707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:32.358707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:12:32.358707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:32.358707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 37/74] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:12:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:32.359352 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b09ad-59ba-4abd-8340-a412ae74f46f None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:32.359582 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b09ad-59ba-4abd-8340-a412ae74f46f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:32.360012 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b09ad-59ba-4abd-8340-a412ae74f46f None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:32.361875 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 38/75] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:32.365031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b09ad-59ba-4abd-8340-a412ae74f46f None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:32.385126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:32.394300 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73479f91-757e-4682-9f51-6d0177f0f404 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:32.429932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228617d3-d843-4d21-8f81-f930003d65f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:32.430582 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228617d3-d843-4d21-8f81-f930003d65f7 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:32.430869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228617d3-d843-4d21-8f81-f930003d65f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:32.431156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228617d3-d843-4d21-8f81-f930003d65f7 None admin] PATH_INFO: `/v3/users/39e61fe22f50447d975033301fd5b706` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:32.432356 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228617d3-d843-4d21-8f81-f930003d65f7 None admin] RBAC: Authorizing `identity:get_user(user_id=39e61fe22f50447d975033301fd5b706)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:32.434217 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228617d3-d843-4d21-8f81-f930003d65f7 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:32.436124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 38/76] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:32 2022] GET /identity/v3/users/39e61fe22f50447d975033301fd5b706 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:32.436820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228617d3-d843-4d21-8f81-f930003d65f7 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:32.477680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b09ad-59ba-4abd-8340-a412ae74f46f None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:32.496811 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:32.521087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fe27d-ef81-43f6-a35a-4fbcdae16443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:32.521733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fe27d-ef81-43f6-a35a-4fbcdae16443 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:32.522019 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fe27d-ef81-43f6-a35a-4fbcdae16443 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:32.522296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fe27d-ef81-43f6-a35a-4fbcdae16443 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:32.533185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0fe27d-ef81-43f6-a35a-4fbcdae16443 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:32.537931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-7d0fe27d-ef81-43f6-a35a-4fbcdae16443 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:32.537931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:32.537931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:32.537931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:32.537931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:32.537931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:32.537931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:32.537931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:32.537931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:32.537931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:32.537931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:32.537931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:32.537931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:32.537931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:32.537931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:32.537931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:32.538810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:32.538810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:32.538810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:32.538810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:32.538810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:32.538810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:32.538810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:32.538810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:32.538810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:32.538810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:32.538810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:32.538810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:32.538810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:32.538810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:32.538810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:32.538810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:32.539579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:32.539579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:32.539579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:32.539579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:32.539579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:32.539579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:32.539579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:32.539579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:32.539579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:32.539579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:32.539579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:32.539579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:32.539579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 39/77] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:32.540261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fe27d-ef81-43f6-a35a-4fbcdae16443 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:32.549242 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228617d3-d843-4d21-8f81-f930003d65f7 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:32.586477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e154b-fbf8-45b3-afbb-4b9644afbc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:32.587188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e154b-fbf8-45b3-afbb-4b9644afbc61 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:32.587380 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e154b-fbf8-45b3-afbb-4b9644afbc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:32.587651 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e154b-fbf8-45b3-afbb-4b9644afbc61 None admin] PATH_INFO: `/v3/users/14ef987ec6c24ababb3cce74ff137945` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:32.602645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e154b-fbf8-45b3-afbb-4b9644afbc61 None admin] RBAC: Authorizing `identity:get_user(user_id=14ef987ec6c24ababb3cce74ff137945)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:32.604245 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e154b-fbf8-45b3-afbb-4b9644afbc61 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:32.606026 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 39/78] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:32 2022] GET /identity/v3/users/14ef987ec6c24ababb3cce74ff137945 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:32.607281 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8646f1-e0e4-49bc-8930-fe3aa222f029 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:32.607579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8646f1-e0e4-49bc-8930-fe3aa222f029 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:32.607855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8646f1-e0e4-49bc-8930-fe3aa222f029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:32.622455 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:32.626913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-ec8646f1-e0e4-49bc-8930-fe3aa222f029 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:32.628176 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-ec8646f1-e0e4-49bc-8930-fe3aa222f029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:32.648276 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fe27d-ef81-43f6-a35a-4fbcdae16443 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:32.684801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c016e3-a0a8-4f33-acc3-73de9e09e191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:32.685558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c016e3-a0a8-4f33-acc3-73de9e09e191 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:32.685838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c016e3-a0a8-4f33-acc3-73de9e09e191 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:32.686109 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c016e3-a0a8-4f33-acc3-73de9e09e191 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:32.686775 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c016e3-a0a8-4f33-acc3-73de9e09e191 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:32.687080 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c016e3-a0a8-4f33-acc3-73de9e09e191 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:32.688525 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c016e3-a0a8-4f33-acc3-73de9e09e191 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:32.696917 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 40/79] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:12:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:32.697744 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c016e3-a0a8-4f33-acc3-73de9e09e191 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:32.707129 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 40/80] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:32.708110 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8646f1-e0e4-49bc-8930-fe3aa222f029 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:32.759400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:32.804310 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c016e3-a0a8-4f33-acc3-73de9e09e191 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:32.817021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8646f1-e0e4-49bc-8930-fe3aa222f029 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:32.838139 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032b1e1-80ed-4a17-9e14-cf20613c2081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:32.838758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032b1e1-80ed-4a17-9e14-cf20613c2081 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:32.839033 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032b1e1-80ed-4a17-9e14-cf20613c2081 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:32.839299 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032b1e1-80ed-4a17-9e14-cf20613c2081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:32.849249 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:32.852740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9b2e2-2b86-404a-bab8-0f414a7a9b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:32.853453 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9b2e2-2b86-404a-bab8-0f414a7a9b47 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:32.853654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9b2e2-2b86-404a-bab8-0f414a7a9b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:32.853927 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9b2e2-2b86-404a-bab8-0f414a7a9b47 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:32.854614 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9b2e2-2b86-404a-bab8-0f414a7a9b47 None admin] RBAC: Adding query filter params (name=member) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:32.854918 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9b2e2-2b86-404a-bab8-0f414a7a9b47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:32.856495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9b2e2-2b86-404a-bab8-0f414a7a9b47 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:32.865258 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 41/81] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:12:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:32.866089 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9b2e2-2b86-404a-bab8-0f414a7a9b47 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:32.914846 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032b1e1-80ed-4a17-9e14-cf20613c2081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:32.916283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032b1e1-80ed-4a17-9e14-cf20613c2081 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:32.974063 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9b2e2-2b86-404a-bab8-0f414a7a9b47 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:32.976652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:33.014050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246d269-be06-451f-be7a-1862dc1c7587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:33.014686 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246d269-be06-451f-be7a-1862dc1c7587 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:33.014972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246d269-be06-451f-be7a-1862dc1c7587 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:33.015250 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246d269-be06-451f-be7a-1862dc1c7587 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:33.020012 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 41/82] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:12:33.020895 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032b1e1-80ed-4a17-9e14-cf20613c2081 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:33.061264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:33.091668 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246d269-be06-451f-be7a-1862dc1c7587 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:33.093428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246d269-be06-451f-be7a-1862dc1c7587 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:33.123929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032b1e1-80ed-4a17-9e14-cf20613c2081 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:33.160899 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2a702-d6cd-45ca-9565-f7dd975a4e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:33.161894 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2a702-d6cd-45ca-9565-f7dd975a4e19 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:33.162330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2a702-d6cd-45ca-9565-f7dd975a4e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:33.162759 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2a702-d6cd-45ca-9565-f7dd975a4e19 None admin] PATH_INFO: `/v3/users/14ef987ec6c24ababb3cce74ff137945` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:33.164151 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2a702-d6cd-45ca-9565-f7dd975a4e19 None admin] RBAC: Authorizing `identity:get_user(user_id=14ef987ec6c24ababb3cce74ff137945)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:33.165635 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2a702-d6cd-45ca-9565-f7dd975a4e19 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:33.167267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 42/83] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:33 2022] GET /identity/v3/users/14ef987ec6c24ababb3cce74ff137945 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:33.168215 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2a702-d6cd-45ca-9565-f7dd975a4e19 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:33.198926 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 42/84] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:12:33.199799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246d269-be06-451f-be7a-1862dc1c7587 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:33.202985 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:33.274827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2a702-d6cd-45ca-9565-f7dd975a4e19 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:33.282235 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:33.303014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246d269-be06-451f-be7a-1862dc1c7587 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:33.314961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc5f69-a37c-4572-8e9a-b002a380da62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:33.315610 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc5f69-a37c-4572-8e9a-b002a380da62 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:33.315894 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc5f69-a37c-4572-8e9a-b002a380da62 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:33.316198 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc5f69-a37c-4572-8e9a-b002a380da62 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:33.317414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc5f69-a37c-4572-8e9a-b002a380da62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:33.318876 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc5f69-a37c-4572-8e9a-b002a380da62 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:33.320521 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 43/85] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:12:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:33.325323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc5f69-a37c-4572-8e9a-b002a380da62 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:33.354110 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4643d9a-6d86-404b-a458-80b75f8622b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:33.354769 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4643d9a-6d86-404b-a458-80b75f8622b1 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:33.355052 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4643d9a-6d86-404b-a458-80b75f8622b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:33.355329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4643d9a-6d86-404b-a458-80b75f8622b1 None admin] PATH_INFO: `/v3/users/14ef987ec6c24ababb3cce74ff137945` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:33.356585 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4643d9a-6d86-404b-a458-80b75f8622b1 None admin] RBAC: Authorizing `identity:get_user(user_id=14ef987ec6c24ababb3cce74ff137945)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:33.358104 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4643d9a-6d86-404b-a458-80b75f8622b1 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:33.359743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 43/86] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:33 2022] GET /identity/v3/users/14ef987ec6c24ababb3cce74ff137945 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:33.360552 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4643d9a-6d86-404b-a458-80b75f8622b1 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:33.428470 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc5f69-a37c-4572-8e9a-b002a380da62 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:33.435343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:33.464111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fb1e0-865c-4df8-89e3-a9a435569b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:33.464769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fb1e0-865c-4df8-89e3-a9a435569b02 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:33.465055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fb1e0-865c-4df8-89e3-a9a435569b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:33.465334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9fb1e0-865c-4df8-89e3-a9a435569b02 None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:33.466533 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9fb1e0-865c-4df8-89e3-a9a435569b02 None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:33.468013 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9fb1e0-865c-4df8-89e3-a9a435569b02 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:33.469880 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 44/87] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:33 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:33.470277 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4643d9a-6d86-404b-a458-80b75f8622b1 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:33.470565 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fb1e0-865c-4df8-89e3-a9a435569b02 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:33.510310 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e2733-02f6-46ae-bef7-8de5c3c51420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:33.511262 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e2733-02f6-46ae-bef7-8de5c3c51420 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:33.511713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e2733-02f6-46ae-bef7-8de5c3c51420 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:33.512206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e2733-02f6-46ae-bef7-8de5c3c51420 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:33.522204 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:33.590776 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9fb1e0-865c-4df8-89e3-a9a435569b02 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:33.596190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e2733-02f6-46ae-bef7-8de5c3c51420 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:33.597704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e2733-02f6-46ae-bef7-8de5c3c51420 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:33.625743 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d20ba-9fb1-4460-b3c9-1afd64d0896f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:33.626369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d20ba-9fb1-4460-b3c9-1afd64d0896f None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:33.626680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d20ba-9fb1-4460-b3c9-1afd64d0896f None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:33.626953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d20ba-9fb1-4460-b3c9-1afd64d0896f None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:33.628206 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d20ba-9fb1-4460-b3c9-1afd64d0896f None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:33.629684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2d20ba-9fb1-4460-b3c9-1afd64d0896f None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:33.631348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 45/88] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:33 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:33.636487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d20ba-9fb1-4460-b3c9-1afd64d0896f None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:33.671726 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:33.707072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 44/89] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:12:33.707866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e2733-02f6-46ae-bef7-8de5c3c51420 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:33.744480 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d20ba-9fb1-4460-b3c9-1afd64d0896f None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:33.755839 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:33.781913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f76d2b-b961-475c-9bc9-65ebd5c05b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:33.782528 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76d2b-b961-475c-9bc9-65ebd5c05b86 None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:33.782811 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76d2b-b961-475c-9bc9-65ebd5c05b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:33.783088 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f76d2b-b961-475c-9bc9-65ebd5c05b86 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:33.783751 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f76d2b-b961-475c-9bc9-65ebd5c05b86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:33.785239 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f76d2b-b961-475c-9bc9-65ebd5c05b86 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:33.826306 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 46/90] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:12:33 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:12:33.826306 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f76d2b-b961-475c-9bc9-65ebd5c05b86 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:33.826578 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e2733-02f6-46ae-bef7-8de5c3c51420 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:33.863395 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa247e0-c811-4c3f-bb80-7a17d1b5b782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:33.864161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa247e0-c811-4c3f-bb80-7a17d1b5b782 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:33.864512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa247e0-c811-4c3f-bb80-7a17d1b5b782 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:33.864816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa247e0-c811-4c3f-bb80-7a17d1b5b782 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:33.925103 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:33.953217 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f76d2b-b961-475c-9bc9-65ebd5c05b86 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:33.956119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa247e0-c811-4c3f-bb80-7a17d1b5b782 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:33.957733 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa247e0-c811-4c3f-bb80-7a17d1b5b782 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:33.995857 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808bfde-1ce9-42b1-86e5-1e5f49c842c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:33.996685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808bfde-1ce9-42b1-86e5-1e5f49c842c2 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:33.997113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808bfde-1ce9-42b1-86e5-1e5f49c842c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:33.997361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0808bfde-1ce9-42b1-86e5-1e5f49c842c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:33.998104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808bfde-1ce9-42b1-86e5-1e5f49c842c2 None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=cabd760392b6495e87b66b85b6a15ca0, user.id=39e61fe22f50447d975033301fd5b706) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:33.998455 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808bfde-1ce9-42b1-86e5-1e5f49c842c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:33.999925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0808bfde-1ce9-42b1-86e5-1e5f49c842c2 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:34.007126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 47/91] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:12:33 2022] GET /identity/v3/role_assignments?user.id=39e61fe22f50447d975033301fd5b706&scope.project.id=cabd760392b6495e87b66b85b6a15ca0&role.id=4da42f08d107448fb877b42b61bc74ea => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:34.008466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808bfde-1ce9-42b1-86e5-1e5f49c842c2 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:34.054534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:34.106144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 45/92] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:12:34.108043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa247e0-c811-4c3f-bb80-7a17d1b5b782 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:34.154508 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0808bfde-1ce9-42b1-86e5-1e5f49c842c2 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:34.211339 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:34.212418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2c7a9-5445-4052-88a7-16fcbd4ed4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:34.213147 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2c7a9-5445-4052-88a7-16fcbd4ed4fa None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:34.213429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2c7a9-5445-4052-88a7-16fcbd4ed4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:34.213782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2c7a9-5445-4052-88a7-16fcbd4ed4fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:34.214569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2c7a9-5445-4052-88a7-16fcbd4ed4fa None admin] RBAC: Adding query filter params (role.id=246bbd6878334bba893bc6f5bf5caee0, scope.project.id=cabd760392b6495e87b66b85b6a15ca0, user.id=14ef987ec6c24ababb3cce74ff137945) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:34.214883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2c7a9-5445-4052-88a7-16fcbd4ed4fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:34.216430 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf2c7a9-5445-4052-88a7-16fcbd4ed4fa None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:34.234087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 48/93] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:12:34 2022] GET /identity/v3/role_assignments?user.id=14ef987ec6c24ababb3cce74ff137945&scope.project.id=cabd760392b6495e87b66b85b6a15ca0&role.id=246bbd6878334bba893bc6f5bf5caee0 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:34.234804 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2c7a9-5445-4052-88a7-16fcbd4ed4fa None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:34.251267 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa247e0-c811-4c3f-bb80-7a17d1b5b782 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:34.292898 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e679378-d265-46cf-9b15-2e2c3d593906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:34.293662 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e679378-d265-46cf-9b15-2e2c3d593906 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:34.293999 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e679378-d265-46cf-9b15-2e2c3d593906 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:34.294285 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e679378-d265-46cf-9b15-2e2c3d593906 None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:34.295499 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e679378-d265-46cf-9b15-2e2c3d593906 None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:34.297107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e679378-d265-46cf-9b15-2e2c3d593906 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:34.298880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 46/94] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:34 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:34.305197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e679378-d265-46cf-9b15-2e2c3d593906 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:34.362905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2c7a9-5445-4052-88a7-16fcbd4ed4fa None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:34.369414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:34.425475 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e679378-d265-46cf-9b15-2e2c3d593906 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:34.432876 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff821e-cbe0-4575-9737-7e278eda513e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:34.434362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff821e-cbe0-4575-9737-7e278eda513e None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:34.434856 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff821e-cbe0-4575-9737-7e278eda513e None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:34.435327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff821e-cbe0-4575-9737-7e278eda513e None admin] PATH_INFO: `/v3/projects/edcd2e6e9a8f4c3ebec542d289cb0c95` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:34.436946 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff821e-cbe0-4575-9737-7e278eda513e None admin] RBAC: Authorizing `identity:get_project(project_id=edcd2e6e9a8f4c3ebec542d289cb0c95)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:34.438680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff821e-cbe0-4575-9737-7e278eda513e None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:34.440759 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 49/95] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:34 2022] GET /identity/v3/projects/edcd2e6e9a8f4c3ebec542d289cb0c95 => generated 312 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:34.446865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff821e-cbe0-4575-9737-7e278eda513e None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:34.470765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ef55a-c67d-41dd-adbb-4fd47a01a85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:34.471324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ef55a-c67d-41dd-adbb-4fd47a01a85a None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:34.471604 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ef55a-c67d-41dd-adbb-4fd47a01a85a None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:34.471883 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ef55a-c67d-41dd-adbb-4fd47a01a85a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:34.472689 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ef55a-c67d-41dd-adbb-4fd47a01a85a None admin] RBAC: Adding query filter params (role.id=9a1c954216e34607a6788f44f07a593b, scope.project.id=cabd760392b6495e87b66b85b6a15ca0, user.id=14ef987ec6c24ababb3cce74ff137945) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:34.472966 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ef55a-c67d-41dd-adbb-4fd47a01a85a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:34.474469 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ef55a-c67d-41dd-adbb-4fd47a01a85a None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:34.488189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 47/96] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:12:34 2022] GET /identity/v3/role_assignments?user.id=14ef987ec6c24ababb3cce74ff137945&scope.project.id=cabd760392b6495e87b66b85b6a15ca0&role.id=9a1c954216e34607a6788f44f07a593b => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:34.602971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff821e-cbe0-4575-9737-7e278eda513e None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:34.640671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb6734-9485-4086-b344-163fcd893231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:34.642179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb6734-9485-4086-b344-163fcd893231 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:34.642670 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb6734-9485-4086-b344-163fcd893231 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:34.643149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb6734-9485-4086-b344-163fcd893231 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:34.644202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb6734-9485-4086-b344-163fcd893231 None admin] RBAC: Adding query filter params (role.id=246bbd6878334bba893bc6f5bf5caee0, scope.project.id=edcd2e6e9a8f4c3ebec542d289cb0c95, user.id=14ef987ec6c24ababb3cce74ff137945) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:34.644706 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb6734-9485-4086-b344-163fcd893231 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:34.646399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb6734-9485-4086-b344-163fcd893231 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:34.653168 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 50/97] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:12:34 2022] GET /identity/v3/role_assignments?user.id=14ef987ec6c24ababb3cce74ff137945&scope.project.id=edcd2e6e9a8f4c3ebec542d289cb0c95&role.id=246bbd6878334bba893bc6f5bf5caee0 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:36.669348 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd0464-f851-40af-bf88-926b260f754b None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:36.669868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd0464-f851-40af-bf88-926b260f754b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:36.669868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dd0464-f851-40af-bf88-926b260f754b None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:36.671244 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 48/98] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:36.677303 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc8bde-aded-4ce2-89cd-bc49f19dd13c None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:36.677724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc8bde-aded-4ce2-89cd-bc49f19dd13c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:36.677894 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc8bde-aded-4ce2-89cd-bc49f19dd13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:36.704960 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-32bc8bde-aded-4ce2-89cd-bc49f19dd13c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:36.706171 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-32bc8bde-aded-4ce2-89cd-bc49f19dd13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:36.726998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:36.773660 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 51/99] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:36.811273 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd0464-f851-40af-bf88-926b260f754b None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:36.815368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:36.913303 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc5e31-4eba-4288-a567-4189261de315 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:36.913633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc5e31-4eba-4288-a567-4189261de315 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:36.913847 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc5e31-4eba-4288-a567-4189261de315 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:36.915475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 52/100] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:36.930103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd075e-8ae5-47b8-ae0d-c2a53ccef33f None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:36.930383 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd075e-8ae5-47b8-ae0d-c2a53ccef33f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:36.930576 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd075e-8ae5-47b8-ae0d-c2a53ccef33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:36.949425 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dd0464-f851-40af-bf88-926b260f754b None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:36.960749 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-c4fd075e-8ae5-47b8-ae0d-c2a53ccef33f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:36.962351 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-c4fd075e-8ae5-47b8-ae0d-c2a53ccef33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:36.999486 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51b624-a180-4edc-b6a4-ac974f6e3d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:37.000469 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51b624-a180-4edc-b6a4-ac974f6e3d84 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:37.000921 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51b624-a180-4edc-b6a4-ac974f6e3d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:37.001364 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51b624-a180-4edc-b6a4-ac974f6e3d84 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:37.019939 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51b624-a180-4edc-b6a4-ac974f6e3d84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:37.026682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-8f51b624-a180-4edc-b6a4-ac974f6e3d84 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:12:37.026682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:37.026682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:37.026682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:37.026682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:37.026682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:37.026682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:37.026682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:37.026682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:37.026682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:37.026682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:37.026682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:37.026682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:37.026682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:37.026682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:37.026682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:37.027562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:37.027562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:37.027562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:37.027562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:37.027562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:37.027562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:37.027562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:37.027562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:37.027562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:37.027562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:37.027562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:37.027562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:37.027562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:37.027562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:37.027562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:37.027562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:37.028427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:37.028427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:37.028427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:37.028427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:37.028427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:37.028427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:37.028427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:37.028427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:37.028427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:37.028427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:37.028427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:12:37.028427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:12:37.029442 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 49/101] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:37.036800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa934513-3770-4fd2-b79e-de94a13f5e43 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:37.037280 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa934513-3770-4fd2-b79e-de94a13f5e43 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:37.037756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa934513-3770-4fd2-b79e-de94a13f5e43 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:37.044150 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 50/102] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:37.045128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa934513-3770-4fd2-b79e-de94a13f5e43 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:37.110234 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:37.124334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 53/103] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:12:37.125799 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998589e2-9571-4b8d-b8de-a917344a5064 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:37.126090 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998589e2-9571-4b8d-b8de-a917344a5064 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:37.126375 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998589e2-9571-4b8d-b8de-a917344a5064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:37.152016 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa934513-3770-4fd2-b79e-de94a13f5e43 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:37.164421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-998589e2-9571-4b8d-b8de-a917344a5064 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:37.165629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-998589e2-9571-4b8d-b8de-a917344a5064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:37.188585 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851df6ea-4228-4070-81ab-dbdbe286300e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:37.189534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851df6ea-4228-4070-81ab-dbdbe286300e None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:37.189986 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851df6ea-4228-4070-81ab-dbdbe286300e None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:37.190480 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851df6ea-4228-4070-81ab-dbdbe286300e None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:37.191682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851df6ea-4228-4070-81ab-dbdbe286300e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:37.192197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851df6ea-4228-4070-81ab-dbdbe286300e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:37.193837 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851df6ea-4228-4070-81ab-dbdbe286300e None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:37.203193 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 51/104] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:12:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:37.204488 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851df6ea-4228-4070-81ab-dbdbe286300e None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:37.309241 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851df6ea-4228-4070-81ab-dbdbe286300e None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:37.337993 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:37.342565 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 54/105] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:37.342565 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998589e2-9571-4b8d-b8de-a917344a5064 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:37.354160 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475bef4e-d3be-4d9c-bdbc-d0c90fed6f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:37.355111 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bef4e-d3be-4d9c-bdbc-d0c90fed6f14 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:37.355585 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bef4e-d3be-4d9c-bdbc-d0c90fed6f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:37.356247 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bef4e-d3be-4d9c-bdbc-d0c90fed6f14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:37.357805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475bef4e-d3be-4d9c-bdbc-d0c90fed6f14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:37.359532 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475bef4e-d3be-4d9c-bdbc-d0c90fed6f14 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:37.361542 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 52/106] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:12:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:37.367835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475bef4e-d3be-4d9c-bdbc-d0c90fed6f14 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:37.381928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:37.532692 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998589e2-9571-4b8d-b8de-a917344a5064 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:37.543956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475bef4e-d3be-4d9c-bdbc-d0c90fed6f14 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:37.578884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf57ab4-adaa-4bff-81be-a11b632440a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:37.580025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf57ab4-adaa-4bff-81be-a11b632440a3 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:37.580153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c1c66-ba2b-41ca-b9b5-e86d8a14e36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:37.580669 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf57ab4-adaa-4bff-81be-a11b632440a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:37.580781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c1c66-ba2b-41ca-b9b5-e86d8a14e36a None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:37.580992 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c1c66-ba2b-41ca-b9b5-e86d8a14e36a None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:37.581262 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c1c66-ba2b-41ca-b9b5-e86d8a14e36a None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:37.581609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf57ab4-adaa-4bff-81be-a11b632440a3 None admin] PATH_INFO: `/v3/users/29d18eba49914276818391057165bf4f` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:37.581908 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c1c66-ba2b-41ca-b9b5-e86d8a14e36a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:37.583373 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c1c66-ba2b-41ca-b9b5-e86d8a14e36a None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:37.601879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf57ab4-adaa-4bff-81be-a11b632440a3 None admin] RBAC: Authorizing `identity:get_user(user_id=29d18eba49914276818391057165bf4f)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:37.603846 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf57ab4-adaa-4bff-81be-a11b632440a3 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:37.606068 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 55/107] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:37 2022] GET /identity/v3/users/29d18eba49914276818391057165bf4f => generated 289 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:37.610639 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf57ab4-adaa-4bff-81be-a11b632440a3 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:37.619396 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 53/108] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Aug 7 19:12:37 2022] POST /identity/v3/projects => generated 302 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:12:37.625460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810290d3-116f-4588-af02-784a00eb6086 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:37.625755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810290d3-116f-4588-af02-784a00eb6086 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:37.626018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810290d3-116f-4588-af02-784a00eb6086 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:37.627577 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 54/109] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:37.632939 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810290d3-116f-4588-af02-784a00eb6086 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:37.726274 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:37.726999 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:37.766725 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf57ab4-adaa-4bff-81be-a11b632440a3 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:37.802503 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810290d3-116f-4588-af02-784a00eb6086 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:37.817528 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc19da5-99f2-4458-9c00-957fee9528f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:37.818839 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc19da5-99f2-4458-9c00-957fee9528f5 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:37.819350 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc19da5-99f2-4458-9c00-957fee9528f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:37.820056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc19da5-99f2-4458-9c00-957fee9528f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:37.835584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc19da5-99f2-4458-9c00-957fee9528f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:37.842338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-0dc19da5-99f2-4458-9c00-957fee9528f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:37.842338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:37.842338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:37.842338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:37.842338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:37.842338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:37.842338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:37.842338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:37.842338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:37.842338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:37.842338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:37.842338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:37.842338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:37.842338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:37.842338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:37.842338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:37.843261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:37.843261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:37.843261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:37.843261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:37.843261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:37.843261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:37.843261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:37.843261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:37.843261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:37.843261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:37.843261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:37.843261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:37.843261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:37.843261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:37.843261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:37.843261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:37.844121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:37.844121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:37.844121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:37.844121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:37.844121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:37.844121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:37.844121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:37.844121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:37.844121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:37.844121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:37.844121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:37.844121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:37.844121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 56/110] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:12:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:37.844774 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfd8a5-0a96-4808-8e14-8b64065d7677 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:37.845094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfd8a5-0a96-4808-8e14-8b64065d7677 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:37.845378 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfd8a5-0a96-4808-8e14-8b64065d7677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:37.855188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56065e9c-046d-46a6-b88f-247eaf30eb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:37.856136 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56065e9c-046d-46a6-b88f-247eaf30eb20 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:37.856598 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56065e9c-046d-46a6-b88f-247eaf30eb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:37.857047 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56065e9c-046d-46a6-b88f-247eaf30eb20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:37.864800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-adbfd8a5-0a96-4808-8e14-8b64065d7677 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:37.866142 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-adbfd8a5-0a96-4808-8e14-8b64065d7677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:37.943467 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 57/111] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:37.946667 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56065e9c-046d-46a6-b88f-247eaf30eb20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:37.948434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56065e9c-046d-46a6-b88f-247eaf30eb20 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:37.949468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc1473-d375-4f50-8d68-d838421f35ca None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:37.949952 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc1473-d375-4f50-8d68-d838421f35ca None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:37.950322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc1473-d375-4f50-8d68-d838421f35ca None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:37.951723 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 58/112] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:37.956412 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc1473-d375-4f50-8d68-d838421f35ca None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:38.014120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:38.046083 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:38.076043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 55/113] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:12:38.077614 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d890a2-6605-4751-9471-00c1ac66a9f3 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:38.078041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d890a2-6605-4751-9471-00c1ac66a9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:38.078454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d890a2-6605-4751-9471-00c1ac66a9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:38.088622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc1473-d375-4f50-8d68-d838421f35ca None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:38.096999 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-f5d890a2-6605-4751-9471-00c1ac66a9f3 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:38.098588 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f5d890a2-6605-4751-9471-00c1ac66a9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:38.124157 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853a30e-0670-4b7b-9ab3-42291d836ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:38.124772 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853a30e-0670-4b7b-9ab3-42291d836ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:38.125053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853a30e-0670-4b7b-9ab3-42291d836ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:38.125326 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853a30e-0670-4b7b-9ab3-42291d836ec9 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:38.125992 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853a30e-0670-4b7b-9ab3-42291d836ec9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:38.126302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853a30e-0670-4b7b-9ab3-42291d836ec9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:38.127681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853a30e-0670-4b7b-9ab3-42291d836ec9 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:38.135726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 59/114] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:38.140901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853a30e-0670-4b7b-9ab3-42291d836ec9 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:38.170651 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 56/115] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:38.171944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3c556-d249-4da8-b6c4-bf39345b9315 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:38.172264 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3c556-d249-4da8-b6c4-bf39345b9315 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:38.172545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3c556-d249-4da8-b6c4-bf39345b9315 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:38.173784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 57/116] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:38.174480 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3c556-d249-4da8-b6c4-bf39345b9315 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:38.252779 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853a30e-0670-4b7b-9ab3-42291d836ec9 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:38.263284 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:38.279091 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:38.288588 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b2d25-d2f5-47ad-821b-4892f30fc15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:38.289226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b2d25-d2f5-47ad-821b-4892f30fc15a None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:38.289504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b2d25-d2f5-47ad-821b-4892f30fc15a None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:38.289774 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b2d25-d2f5-47ad-821b-4892f30fc15a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:38.290141 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3c556-d249-4da8-b6c4-bf39345b9315 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:38.301078 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b2d25-d2f5-47ad-821b-4892f30fc15a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:38.306159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-8f8b2d25-d2f5-47ad-821b-4892f30fc15a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:38.306159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:38.306159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:38.306159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:38.306159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:38.306159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:38.306159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:38.306159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:38.306159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:38.306159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:38.306159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:38.306159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:38.306159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:38.306159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:38.306159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:38.306159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:38.306971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:38.306971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:38.306971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:38.306971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:38.306971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:38.306971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:38.306971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:38.306971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:38.306971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:38.306971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:38.306971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:38.306971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:38.306971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:38.306971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:38.306971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:38.306971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:38.313187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:38.313187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:38.313187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:38.313187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:38.313187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:38.313187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:38.313187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:38.313187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:38.313187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:38.313187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:38.313187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:38.313187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:38.313187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 60/117] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:38.313187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b2d25-d2f5-47ad-821b-4892f30fc15a None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:38.332612 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d0a40-a646-4d05-8a70-e8a7c8b951af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:38.333260 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d0a40-a646-4d05-8a70-e8a7c8b951af None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:38.333609 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d0a40-a646-4d05-8a70-e8a7c8b951af None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:38.333947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d0a40-a646-4d05-8a70-e8a7c8b951af None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:38.335345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d0a40-a646-4d05-8a70-e8a7c8b951af None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:38.336831 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d0a40-a646-4d05-8a70-e8a7c8b951af None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:38.338625 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 58/118] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:38 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:38.339595 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740014f2-cf2a-433b-8642-68b1d26d2b4d None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:38.339875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740014f2-cf2a-433b-8642-68b1d26d2b4d None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:38.340171 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740014f2-cf2a-433b-8642-68b1d26d2b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:38.359548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-740014f2-cf2a-433b-8642-68b1d26d2b4d None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:38.361008 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-740014f2-cf2a-433b-8642-68b1d26d2b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:38.424030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b2d25-d2f5-47ad-821b-4892f30fc15a None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:38.430176 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 59/119] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:38.431242 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740014f2-cf2a-433b-8642-68b1d26d2b4d None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:38.476823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e39e2-c5b3-44cf-b81d-909c4c87e788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:38.477477 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e39e2-c5b3-44cf-b81d-909c4c87e788 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:38.477765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e39e2-c5b3-44cf-b81d-909c4c87e788 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:38.478048 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e39e2-c5b3-44cf-b81d-909c4c87e788 None admin] PATH_INFO: `/v3/users/39e61fe22f50447d975033301fd5b706` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:38.479193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e39e2-c5b3-44cf-b81d-909c4c87e788 None admin] RBAC: Authorizing `identity:get_user(user_id=39e61fe22f50447d975033301fd5b706)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:38.481244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e39e2-c5b3-44cf-b81d-909c4c87e788 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:38.483558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 61/120] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:38 2022] GET /identity/v3/users/39e61fe22f50447d975033301fd5b706 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:38.484183 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e39e2-c5b3-44cf-b81d-909c4c87e788 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:38.523876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:38.543975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:38.553406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740014f2-cf2a-433b-8642-68b1d26d2b4d None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:38.592101 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e39e2-c5b3-44cf-b81d-909c4c87e788 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:38.593972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f215f-39a4-4d25-8879-746de4009de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:38.594912 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f215f-39a4-4d25-8879-746de4009de0 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:38.594912 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f215f-39a4-4d25-8879-746de4009de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:38.595189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f215f-39a4-4d25-8879-746de4009de0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:38.610383 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f215f-39a4-4d25-8879-746de4009de0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:38.619446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-859f215f-39a4-4d25-8879-746de4009de0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:12:38.619446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:38.619446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:38.619446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:38.619446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:38.619446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:38.619446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:38.619446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:38.619446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:38.619446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:38.619446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:38.619446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:38.619446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:38.619446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:38.619446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:38.619446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:38.620505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:38.620505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:38.620505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:38.620505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:38.620505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:38.620505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:38.620505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:38.620505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:38.620505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:38.620505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:38.620505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:38.620505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:38.620505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:38.620505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:38.620505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:38.620505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:38.626642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:38.626642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:38.626642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:38.626642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:38.626642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:38.626642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:38.626642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:38.626642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:38.626642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:38.626642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:38.626642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:12:38.626642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:12:38.626642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 60/121] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:12:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:38.626642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f215f-39a4-4d25-8879-746de4009de0 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:38.632988 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b843b-31db-4f99-a9c5-8fee393b96a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:38.633618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b843b-31db-4f99-a9c5-8fee393b96a2 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:38.633895 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b843b-31db-4f99-a9c5-8fee393b96a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:38.634167 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b843b-31db-4f99-a9c5-8fee393b96a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:38.634840 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b843b-31db-4f99-a9c5-8fee393b96a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:38.635373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b843b-31db-4f99-a9c5-8fee393b96a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:38.636585 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b843b-31db-4f99-a9c5-8fee393b96a2 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:38.646429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 62/122] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:12:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:38.647019 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b843b-31db-4f99-a9c5-8fee393b96a2 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:38.732256 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f215f-39a4-4d25-8879-746de4009de0 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:38.756365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b843b-31db-4f99-a9c5-8fee393b96a2 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:38.768362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20a454-47b7-4a5a-bb54-1da3b534e193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:38.768981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20a454-47b7-4a5a-bb54-1da3b534e193 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:38.769259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20a454-47b7-4a5a-bb54-1da3b534e193 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:38.769835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e20a454-47b7-4a5a-bb54-1da3b534e193 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:38.770259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20a454-47b7-4a5a-bb54-1da3b534e193 None admin] RBAC: Adding query filter params (role.id=31d708bdce024bbc8423f085d27b1776, scope.project.id=cabd760392b6495e87b66b85b6a15ca0, user.id=29d18eba49914276818391057165bf4f) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:38.770564 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20a454-47b7-4a5a-bb54-1da3b534e193 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:38.772214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e20a454-47b7-4a5a-bb54-1da3b534e193 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:38.773424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:38.775427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:38.788652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 61/123] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:12:38 2022] GET /identity/v3/role_assignments?user.id=29d18eba49914276818391057165bf4f&scope.project.id=cabd760392b6495e87b66b85b6a15ca0&role.id=31d708bdce024bbc8423f085d27b1776 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:38.788652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20a454-47b7-4a5a-bb54-1da3b534e193 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:38.795105 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ea7d9-0333-4f27-9319-c987c0462324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:38.795766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ea7d9-0333-4f27-9319-c987c0462324 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:38.796081 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ea7d9-0333-4f27-9319-c987c0462324 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:38.796363 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8ea7d9-0333-4f27-9319-c987c0462324 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:38.809721 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8ea7d9-0333-4f27-9319-c987c0462324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:38.815192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-fb8ea7d9-0333-4f27-9319-c987c0462324 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:38.815192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:38.815192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:38.815192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:38.815192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:38.815192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:38.815192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:38.815192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:38.815192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:38.815192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:38.815192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:38.815192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:38.815192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:38.815192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:38.815192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:38.815192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:38.816163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:38.816163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:38.816163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:38.816163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:38.816163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:38.816163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:38.816163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:38.816163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:38.816163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:38.816163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:38.816163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:38.816163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:38.816163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:38.816163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:38.816163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:38.816163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:38.816944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:38.816944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:38.816944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:38.816944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:38.816944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:38.816944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:38.816944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:38.816944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:38.816944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:38.816944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:38.816944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:38.816944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:38.816944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 63/124] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:38.818208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ea7d9-0333-4f27-9319-c987c0462324 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:38.935975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8ea7d9-0333-4f27-9319-c987c0462324 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:38.937454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e20a454-47b7-4a5a-bb54-1da3b534e193 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:38.981149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398031f-0228-4248-8a96-5510a7db22b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:38.982779 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398031f-0228-4248-8a96-5510a7db22b8 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:38.982779 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398031f-0228-4248-8a96-5510a7db22b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:38.982779 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398031f-0228-4248-8a96-5510a7db22b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:38.982986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398031f-0228-4248-8a96-5510a7db22b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:38.983186 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398031f-0228-4248-8a96-5510a7db22b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:38.984632 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398031f-0228-4248-8a96-5510a7db22b8 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:38.987697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c8634-2077-42c0-822e-8d8725cc6b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:38.988374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c8634-2077-42c0-822e-8d8725cc6b86 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:38.988648 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c8634-2077-42c0-822e-8d8725cc6b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:38.988924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7c8634-2077-42c0-822e-8d8725cc6b86 None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:38.990204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c8634-2077-42c0-822e-8d8725cc6b86 None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:38.992286 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7c8634-2077-42c0-822e-8d8725cc6b86 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:38.994416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 62/125] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:38 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:38.995013 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c8634-2077-42c0-822e-8d8725cc6b86 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:39.001426 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 64/126] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:12:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:39.003291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398031f-0228-4248-8a96-5510a7db22b8 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:39.053218 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:39.116947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398031f-0228-4248-8a96-5510a7db22b8 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:39.120458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:39.131125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7c8634-2077-42c0-822e-8d8725cc6b86 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:39.161765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a82519-7d20-45d4-9176-880a6504c87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:39.163199 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a82519-7d20-45d4-9176-880a6504c87a None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:39.163652 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a82519-7d20-45d4-9176-880a6504c87a None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:39.164130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a82519-7d20-45d4-9176-880a6504c87a None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:39.165385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a82519-7d20-45d4-9176-880a6504c87a None admin] RBAC: Adding query filter params (name=member) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:39.165974 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a82519-7d20-45d4-9176-880a6504c87a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:39.167856 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a82519-7d20-45d4-9176-880a6504c87a None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:39.170183 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42769f4c-c71d-4b2c-add7-b8b1b42f5f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:39.170993 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42769f4c-c71d-4b2c-add7-b8b1b42f5f85 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:39.171345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42769f4c-c71d-4b2c-add7-b8b1b42f5f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:39.171690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42769f4c-c71d-4b2c-add7-b8b1b42f5f85 None admin] PATH_INFO: `/v3/users/14ef987ec6c24ababb3cce74ff137945` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:39.172930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42769f4c-c71d-4b2c-add7-b8b1b42f5f85 None admin] RBAC: Authorizing `identity:get_user(user_id=14ef987ec6c24ababb3cce74ff137945)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:39.174905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42769f4c-c71d-4b2c-add7-b8b1b42f5f85 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:39.176357 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 63/127] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:38 2022] GET /identity/v3/users/14ef987ec6c24ababb3cce74ff137945 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:39.177553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42769f4c-c71d-4b2c-add7-b8b1b42f5f85 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:39.178605 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 65/128] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:12:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:39.185596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a82519-7d20-45d4-9176-880a6504c87a None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:39.298645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42769f4c-c71d-4b2c-add7-b8b1b42f5f85 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:39.304860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:39.307912 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a82519-7d20-45d4-9176-880a6504c87a None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:39.347356 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7308bf-558b-4778-8611-acecbc6963c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:39.348161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7308bf-558b-4778-8611-acecbc6963c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:39.348520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7308bf-558b-4778-8611-acecbc6963c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:39.348913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7308bf-558b-4778-8611-acecbc6963c6 None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0/users/39e61fe22f50447d975033301fd5b706/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:39.350698 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf58456-d676-424d-a19f-c4a6b59eb5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:39.351181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7308bf-558b-4778-8611-acecbc6963c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=cabd760392b6495e87b66b85b6a15ca0, user_id=39e61fe22f50447d975033301fd5b706, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:39.351723 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf58456-d676-424d-a19f-c4a6b59eb5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:39.352194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf58456-d676-424d-a19f-c4a6b59eb5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:39.352627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf58456-d676-424d-a19f-c4a6b59eb5b7 None admin] PATH_INFO: `/v3/users/14ef987ec6c24ababb3cce74ff137945` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:39.352951 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7308bf-558b-4778-8611-acecbc6963c6 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:39.354125 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf58456-d676-424d-a19f-c4a6b59eb5b7 None admin] RBAC: Authorizing `identity:get_user(user_id=14ef987ec6c24ababb3cce74ff137945)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:39.355554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:39.357811 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf58456-d676-424d-a19f-c4a6b59eb5b7 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:39.360193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 66/129] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:39 2022] GET /identity/v3/users/14ef987ec6c24ababb3cce74ff137945 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:39.360920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf58456-d676-424d-a19f-c4a6b59eb5b7 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:39.373280 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 64/130] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:12:39 2022] PUT /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0/users/39e61fe22f50447d975033301fd5b706/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:12:39.374216 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7308bf-558b-4778-8611-acecbc6963c6 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:39.471927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf58456-d676-424d-a19f-c4a6b59eb5b7 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:39.495034 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7308bf-558b-4778-8611-acecbc6963c6 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:39.509743 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f27c52-0dfb-43a6-9f93-8c43c03b9ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:39.510851 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f27c52-0dfb-43a6-9f93-8c43c03b9ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:39.511376 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f27c52-0dfb-43a6-9f93-8c43c03b9ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:39.511873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f27c52-0dfb-43a6-9f93-8c43c03b9ec5 None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:39.513526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f27c52-0dfb-43a6-9f93-8c43c03b9ec5 None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:39.515376 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f27c52-0dfb-43a6-9f93-8c43c03b9ec5 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:39.517841 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 67/131] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:39 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:39.519018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f27c52-0dfb-43a6-9f93-8c43c03b9ec5 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:39.534012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfb713-1795-407f-a9e2-ea0bb0a119c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:39.534733 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfb713-1795-407f-a9e2-ea0bb0a119c3 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:39.535013 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfb713-1795-407f-a9e2-ea0bb0a119c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:39.535292 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfb713-1795-407f-a9e2-ea0bb0a119c3 None admin] PATH_INFO: `/v3/users/14ef987ec6c24ababb3cce74ff137945` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:39.536730 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acfb713-1795-407f-a9e2-ea0bb0a119c3 None admin] RBAC: Authorizing `identity:get_user(user_id=14ef987ec6c24ababb3cce74ff137945)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:39.538431 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acfb713-1795-407f-a9e2-ea0bb0a119c3 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:39.540487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 65/132] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:39 2022] GET /identity/v3/users/14ef987ec6c24ababb3cce74ff137945 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:39.541203 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfb713-1795-407f-a9e2-ea0bb0a119c3 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:39.561100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:39.618617 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:39.635947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f27c52-0dfb-43a6-9f93-8c43c03b9ec5 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:39.667539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfb713-1795-407f-a9e2-ea0bb0a119c3 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:39.678506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43093c-7c8b-4fae-af5d-26a467349b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:39.679748 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43093c-7c8b-4fae-af5d-26a467349b30 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:39.680319 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43093c-7c8b-4fae-af5d-26a467349b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:39.680852 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43093c-7c8b-4fae-af5d-26a467349b30 None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:39.682395 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43093c-7c8b-4fae-af5d-26a467349b30 None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:39.684740 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43093c-7c8b-4fae-af5d-26a467349b30 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:39.686954 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 68/133] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:39 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:39.688266 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43093c-7c8b-4fae-af5d-26a467349b30 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:39.712062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5ebb2-577a-4a0a-83fa-2f99aedf4f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:39.712062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5ebb2-577a-4a0a-83fa-2f99aedf4f7b None admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:39.712062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5ebb2-577a-4a0a-83fa-2f99aedf4f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:39.712062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca5ebb2-577a-4a0a-83fa-2f99aedf4f7b None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0/users/14ef987ec6c24ababb3cce74ff137945/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:39.712597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5ebb2-577a-4a0a-83fa-2f99aedf4f7b None admin] RBAC: Authorizing `identity:create_grant(project_id=cabd760392b6495e87b66b85b6a15ca0, user_id=14ef987ec6c24ababb3cce74ff137945, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:39.713096 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca5ebb2-577a-4a0a-83fa-2f99aedf4f7b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:39.732179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 66/134] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:12:39 2022] PUT /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0/users/14ef987ec6c24ababb3cce74ff137945/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:12:39.734086 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5ebb2-577a-4a0a-83fa-2f99aedf4f7b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:39.819081 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43093c-7c8b-4fae-af5d-26a467349b30 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:39.822702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:39.868920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32616e7d-3597-47af-895e-5038ba513174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:39.869642 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32616e7d-3597-47af-895e-5038ba513174 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:39.869928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32616e7d-3597-47af-895e-5038ba513174 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:39.870210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32616e7d-3597-47af-895e-5038ba513174 None admin] PATH_INFO: `/v3/projects/edcd2e6e9a8f4c3ebec542d289cb0c95` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:39.871591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32616e7d-3597-47af-895e-5038ba513174 None admin] RBAC: Authorizing `identity:get_project(project_id=edcd2e6e9a8f4c3ebec542d289cb0c95)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:39.873297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32616e7d-3597-47af-895e-5038ba513174 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:39.875146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca5ebb2-577a-4a0a-83fa-2f99aedf4f7b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:39.875291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 69/135] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:39 2022] GET /identity/v3/projects/edcd2e6e9a8f4c3ebec542d289cb0c95 => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:39.894858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32616e7d-3597-47af-895e-5038ba513174 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:39.909981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:39.926403 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200a65c-2278-414f-9d52-56e051280698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:39.927723 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200a65c-2278-414f-9d52-56e051280698 None admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:39.928089 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200a65c-2278-414f-9d52-56e051280698 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:39.928377 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200a65c-2278-414f-9d52-56e051280698 None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0/users/14ef987ec6c24ababb3cce74ff137945/roles/9a1c954216e34607a6788f44f07a593b` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:39.931109 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200a65c-2278-414f-9d52-56e051280698 None admin] RBAC: Authorizing `identity:create_grant(project_id=cabd760392b6495e87b66b85b6a15ca0, user_id=14ef987ec6c24ababb3cce74ff137945, role_id=9a1c954216e34607a6788f44f07a593b)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:39.933982 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200a65c-2278-414f-9d52-56e051280698 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:39.952745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 67/136] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:12:39 2022] PUT /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0/users/14ef987ec6c24ababb3cce74ff137945/roles/9a1c954216e34607a6788f44f07a593b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:12:40.068444 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32616e7d-3597-47af-895e-5038ba513174 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:40.104381 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc83b8e-ab41-4d8b-bbe7-5d1dd2af191f None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:40.104700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc83b8e-ab41-4d8b-bbe7-5d1dd2af191f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:40.104962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc83b8e-ab41-4d8b-bbe7-5d1dd2af191f None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:40.106853 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 68/137] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:12:40.113852 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0f42c-489d-421e-a56c-4e44d5654109 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:40.114385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0f42c-489d-421e-a56c-4e44d5654109 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:40.114678 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0f42c-489d-421e-a56c-4e44d5654109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:40.142507 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363a981-74fc-4dbf-bb57-fa3347f7916d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:40.143248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363a981-74fc-4dbf-bb57-fa3347f7916d None admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:40.143525 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363a981-74fc-4dbf-bb57-fa3347f7916d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:40.143891 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363a981-74fc-4dbf-bb57-fa3347f7916d None admin] PATH_INFO: `/v3/projects/edcd2e6e9a8f4c3ebec542d289cb0c95/users/14ef987ec6c24ababb3cce74ff137945/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:40.144422 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-92e0f42c-489d-421e-a56c-4e44d5654109 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:40.145798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-92e0f42c-489d-421e-a56c-4e44d5654109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:40.149701 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363a981-74fc-4dbf-bb57-fa3347f7916d None admin] RBAC: Authorizing `identity:create_grant(project_id=edcd2e6e9a8f4c3ebec542d289cb0c95, user_id=14ef987ec6c24ababb3cce74ff137945, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:40.151299 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363a981-74fc-4dbf-bb57-fa3347f7916d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:40.178434 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 70/138] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:12:39 2022] PUT /identity/v3/projects/edcd2e6e9a8f4c3ebec542d289cb0c95/users/14ef987ec6c24ababb3cce74ff137945/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:12:40.242750 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 69/139] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:40.279415 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363a981-74fc-4dbf-bb57-fa3347f7916d None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:40.326010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:40.401419 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363a981-74fc-4dbf-bb57-fa3347f7916d None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:40.444868 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d2160-7ebd-4d62-b01d-8d452103505d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:40.445520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d2160-7ebd-4d62-b01d-8d452103505d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:40.445805 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d2160-7ebd-4d62-b01d-8d452103505d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:40.446084 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d2160-7ebd-4d62-b01d-8d452103505d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:40.447689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d2160-7ebd-4d62-b01d-8d452103505d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:40.449218 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d2160-7ebd-4d62-b01d-8d452103505d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:40.450935 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 71/140] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:12:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:40.456662 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0f42c-489d-421e-a56c-4e44d5654109 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:40.579160 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0f42c-489d-421e-a56c-4e44d5654109 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:40.593377 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:40.622865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5903d-1b78-41f3-93ce-ebe1b1b7cd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:40.623507 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5903d-1b78-41f3-93ce-ebe1b1b7cd62 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:40.623781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5903d-1b78-41f3-93ce-ebe1b1b7cd62 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:40.624097 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5903d-1b78-41f3-93ce-ebe1b1b7cd62 None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:40.624757 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5903d-1b78-41f3-93ce-ebe1b1b7cd62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:40.626233 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5903d-1b78-41f3-93ce-ebe1b1b7cd62 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:40.665988 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 70/141] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:12:40 2022] POST /identity/v3/users => generated 283 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:12:41.601711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd475dd-47d8-4dc3-bbd8-ac527ecfda1d None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:41.602436 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd475dd-47d8-4dc3-bbd8-ac527ecfda1d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:41.602436 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd475dd-47d8-4dc3-bbd8-ac527ecfda1d None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:41.603619 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 72/142] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:41.609059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79431a1d-b377-4604-b503-8d76fd2a2f68 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:41.609606 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79431a1d-b377-4604-b503-8d76fd2a2f68 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:41.609606 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79431a1d-b377-4604-b503-8d76fd2a2f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:41.632798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-79431a1d-b377-4604-b503-8d76fd2a2f68 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:41.633997 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-79431a1d-b377-4604-b503-8d76fd2a2f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:41.722240 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 71/143] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:41.756565 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd475dd-47d8-4dc3-bbd8-ac527ecfda1d None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:41.874021 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:41.877838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd475dd-47d8-4dc3-bbd8-ac527ecfda1d None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:41.922149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181dd80-5e54-45f7-8489-9567b6a264b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:41.922830 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181dd80-5e54-45f7-8489-9567b6a264b5 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:41.923108 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181dd80-5e54-45f7-8489-9567b6a264b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:41.923379 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181dd80-5e54-45f7-8489-9567b6a264b5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:41.940991 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181dd80-5e54-45f7-8489-9567b6a264b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:41.945904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-c181dd80-5e54-45f7-8489-9567b6a264b5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:12:41.945904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:41.945904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:41.945904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:41.945904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:41.945904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:41.945904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:41.945904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:41.945904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:41.945904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:41.945904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:41.945904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:41.945904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:41.945904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:41.945904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:41.945904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:41.948256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:41.948256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:41.948256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:41.948256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:41.948256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:41.948256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:41.948256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:41.948256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:41.948256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:41.948256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:41.948256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:41.948256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:41.948256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:41.948256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:41.948256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:41.948256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:41.950322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:41.950322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:41.950322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:41.950322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:41.950322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:41.950322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:41.950322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:41.950322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:41.950322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:41.950322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:41.950322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:12:41.950322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:41.950322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 73/144] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:41.955861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79431a1d-b377-4604-b503-8d76fd2a2f68 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:42.065039 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:42.069569 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79431a1d-b377-4604-b503-8d76fd2a2f68 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:42.108277 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e826c63-e836-4356-9acb-195181d27cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:42.108911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e826c63-e836-4356-9acb-195181d27cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:42.109197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e826c63-e836-4356-9acb-195181d27cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:42.109474 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e826c63-e836-4356-9acb-195181d27cd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:42.110162 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e826c63-e836-4356-9acb-195181d27cd8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:42.110474 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e826c63-e836-4356-9acb-195181d27cd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:42.111894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e826c63-e836-4356-9acb-195181d27cd8 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:42.124188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 72/145] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:12:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:42.129864 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181dd80-5e54-45f7-8489-9567b6a264b5 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:42.250057 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181dd80-5e54-45f7-8489-9567b6a264b5 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:42.289853 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222b893-d99d-480c-93be-caea2f375dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:42.290504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222b893-d99d-480c-93be-caea2f375dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:42.290785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222b893-d99d-480c-93be-caea2f375dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:42.291060 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222b893-d99d-480c-93be-caea2f375dc6 None admin] PATH_INFO: `/v3/users/29d18eba49914276818391057165bf4f` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:42.292266 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222b893-d99d-480c-93be-caea2f375dc6 None admin] RBAC: Authorizing `identity:get_user(user_id=29d18eba49914276818391057165bf4f)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:42.293767 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222b893-d99d-480c-93be-caea2f375dc6 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:42.295363 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 74/146] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:42 2022] GET /identity/v3/users/29d18eba49914276818391057165bf4f => generated 289 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:42.301175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e826c63-e836-4356-9acb-195181d27cd8 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:42.419328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e826c63-e836-4356-9acb-195181d27cd8 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:42.459532 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b273477-ccbd-4947-8aed-afff06b46afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:42.460218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b273477-ccbd-4947-8aed-afff06b46afc None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:42.460502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b273477-ccbd-4947-8aed-afff06b46afc None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:42.460775 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b273477-ccbd-4947-8aed-afff06b46afc None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:42.461983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b273477-ccbd-4947-8aed-afff06b46afc None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:42.463477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b273477-ccbd-4947-8aed-afff06b46afc None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:42.465234 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 73/147] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:42 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:42.471244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222b893-d99d-480c-93be-caea2f375dc6 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:42.506519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:42.585979 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222b893-d99d-480c-93be-caea2f375dc6 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:42.627746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31a332-b83f-41f9-bdd6-8c7c882d3048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:42.628554 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31a332-b83f-41f9-bdd6-8c7c882d3048 None admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:42.628735 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31a332-b83f-41f9-bdd6-8c7c882d3048 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:42.629015 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31a332-b83f-41f9-bdd6-8c7c882d3048 None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0/users/29d18eba49914276818391057165bf4f/roles/31d708bdce024bbc8423f085d27b1776` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:42.630999 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31a332-b83f-41f9-bdd6-8c7c882d3048 None admin] RBAC: Authorizing `identity:create_grant(project_id=cabd760392b6495e87b66b85b6a15ca0, user_id=29d18eba49914276818391057165bf4f, role_id=31d708bdce024bbc8423f085d27b1776)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:42.632524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31a332-b83f-41f9-bdd6-8c7c882d3048 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:42.652435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad42131-7691-44ec-b5d5-daaa667983f8 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:42.653243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad42131-7691-44ec-b5d5-daaa667983f8 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:42.653243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad42131-7691-44ec-b5d5-daaa667983f8 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:42.654791 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 74/148] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:42.661245 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f377e-67e5-417e-b5ba-31fc0e6b2c53 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:42.661531 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f377e-67e5-417e-b5ba-31fc0e6b2c53 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:42.661814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f377e-67e5-417e-b5ba-31fc0e6b2c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:42.682012 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 75/149] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:12:42 2022] PUT /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0/users/29d18eba49914276818391057165bf4f/roles/31d708bdce024bbc8423f085d27b1776 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:12:42.693311 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-be7f377e-67e5-417e-b5ba-31fc0e6b2c53 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:42.694553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-be7f377e-67e5-417e-b5ba-31fc0e6b2c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:42.730726 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:42.786361 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 75/150] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:12:42.829407 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31a332-b83f-41f9-bdd6-8c7c882d3048 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:42.953499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:42.970077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31a332-b83f-41f9-bdd6-8c7c882d3048 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:43.027742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b75664-f276-43f5-a3bd-2df1441c58e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:43.028497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b75664-f276-43f5-a3bd-2df1441c58e0 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:43.028781 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b75664-f276-43f5-a3bd-2df1441c58e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:43.029053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b75664-f276-43f5-a3bd-2df1441c58e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:43.048390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b75664-f276-43f5-a3bd-2df1441c58e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:43.053599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-d4b75664-f276-43f5-a3bd-2df1441c58e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:43.053599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:43.053599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:43.053599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:43.053599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:43.053599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:43.053599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:43.053599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:43.053599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:43.053599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:43.053599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:43.053599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:43.053599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:43.053599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:43.053599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:43.053599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:43.056654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:43.056654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:43.056654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:43.056654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:43.056654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:43.056654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:43.056654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:43.056654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:43.056654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:43.056654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:43.056654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:43.056654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:43.056654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:43.056654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:43.056654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:43.056654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:43.061551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:43.061551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:43.061551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:43.061551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:43.061551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:43.061551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:43.061551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:43.061551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:43.061551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:43.061551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:43.061551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:43.061551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:43.061551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 76/151] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:12:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:43.063258 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f377e-67e5-417e-b5ba-31fc0e6b2c53 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:43.198742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f377e-67e5-417e-b5ba-31fc0e6b2c53 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:43.243520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a1db2-e198-4796-9ffa-0b9318b99598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:43.244102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a1db2-e198-4796-9ffa-0b9318b99598 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:43.244389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a1db2-e198-4796-9ffa-0b9318b99598 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:43.244664 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a1db2-e198-4796-9ffa-0b9318b99598 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:43.245348 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a1db2-e198-4796-9ffa-0b9318b99598 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:43.245652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a1db2-e198-4796-9ffa-0b9318b99598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:43.247090 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a1db2-e198-4796-9ffa-0b9318b99598 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:43.256042 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 76/152] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:43.261306 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b75664-f276-43f5-a3bd-2df1441c58e0 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:43.268751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ec7d4-0cea-4b89-8efe-dae41f0472ec None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:43.269039 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ec7d4-0cea-4b89-8efe-dae41f0472ec None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:43.269314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ec7d4-0cea-4b89-8efe-dae41f0472ec None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:43.270853 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 77/153] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:12:43.280945 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d6566-5205-447b-8d35-e725b715f445 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:43.281224 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d6566-5205-447b-8d35-e725b715f445 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:43.281490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d6566-5205-447b-8d35-e725b715f445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:43.312998 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-1d2d6566-5205-447b-8d35-e725b715f445 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:43.314209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-1d2d6566-5205-447b-8d35-e725b715f445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:43.339696 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:43.413139 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b75664-f276-43f5-a3bd-2df1441c58e0 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:43.437379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 78/154] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 07 19:12:43.438770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db436937-18ac-46bd-906b-657cd1483c7d None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:43.439054 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db436937-18ac-46bd-906b-657cd1483c7d None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:43.439329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db436937-18ac-46bd-906b-657cd1483c7d None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:43.440934 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 79/155] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:43.446652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db082e-f68c-4f34-ba3c-ebe11376e504 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:43.446942 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db082e-f68c-4f34-ba3c-ebe11376e504 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:43.447193 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db082e-f68c-4f34-ba3c-ebe11376e504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:43.467824 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf12f6-080b-41a1-9af1-60a57c3785a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:43.468516 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf12f6-080b-41a1-9af1-60a57c3785a7 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:43.468797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf12f6-080b-41a1-9af1-60a57c3785a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:43.469077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf12f6-080b-41a1-9af1-60a57c3785a7 None admin] PATH_INFO: `/v3/users/39e61fe22f50447d975033301fd5b706` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:43.472677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf12f6-080b-41a1-9af1-60a57c3785a7 None admin] RBAC: Authorizing `identity:get_user(user_id=39e61fe22f50447d975033301fd5b706)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:43.474160 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf12f6-080b-41a1-9af1-60a57c3785a7 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:43.475784 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 77/156] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:43 2022] GET /identity/v3/users/39e61fe22f50447d975033301fd5b706 => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:43.479901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-a7db082e-f68c-4f34-ba3c-ebe11376e504 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:43.481179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf12f6-080b-41a1-9af1-60a57c3785a7 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:43.481422 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a7db082e-f68c-4f34-ba3c-ebe11376e504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:43.506990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:43.594262 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 80/157] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:43.595212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db082e-f68c-4f34-ba3c-ebe11376e504 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:43.642281 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf12f6-080b-41a1-9af1-60a57c3785a7 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:43.705721 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd2f51-2d52-447c-bdb9-4cb63dff769c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:43.706369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd2f51-2d52-447c-bdb9-4cb63dff769c None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:43.706654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd2f51-2d52-447c-bdb9-4cb63dff769c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:43.706918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd2f51-2d52-447c-bdb9-4cb63dff769c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:43.722392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:43.741388 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db082e-f68c-4f34-ba3c-ebe11376e504 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:43.796316 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644ca79-3054-4d7c-b886-c911b92c3b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:43.796882 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644ca79-3054-4d7c-b886-c911b92c3b98 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:43.797159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644ca79-3054-4d7c-b886-c911b92c3b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:43.797534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644ca79-3054-4d7c-b886-c911b92c3b98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:43.815303 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644ca79-3054-4d7c-b886-c911b92c3b98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:43.815978 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd2f51-2d52-447c-bdb9-4cb63dff769c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:43.820106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-b644ca79-3054-4d7c-b886-c911b92c3b98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:43.820106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:43.820106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:43.820106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:43.820106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:43.820106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:43.820106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:43.820106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:43.820106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:43.820106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:43.820106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:43.820106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:43.820106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:43.820106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:43.820106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:43.820106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:43.821641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:43.821641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:43.821641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:43.821641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:43.821641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:43.821641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:43.821641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:43.821641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:43.821641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:43.821641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:43.821641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:43.821641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:43.821641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:43.821641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:43.821641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:43.821641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:43.822973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:43.822973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:43.822973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:43.822973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:43.822973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:43.822973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:43.822973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:43.822973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:43.822973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:43.822973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:43.822973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:43.822973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:12:43.822973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 81/158] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:43.822973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644ca79-3054-4d7c-b886-c911b92c3b98 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:43.824183 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd2f51-2d52-447c-bdb9-4cb63dff769c None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:43.882194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:44.048238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644ca79-3054-4d7c-b886-c911b92c3b98 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:44.122417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4451af3-d24c-49f1-81f1-aa69ff1467ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:44.123126 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4451af3-d24c-49f1-81f1-aa69ff1467ac None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:44.123409 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4451af3-d24c-49f1-81f1-aa69ff1467ac None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:44.123739 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4451af3-d24c-49f1-81f1-aa69ff1467ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:44.144163 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:44.145950 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4451af3-d24c-49f1-81f1-aa69ff1467ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:44.151037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-f4451af3-d24c-49f1-81f1-aa69ff1467ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:12:44.151037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:44.151037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:44.151037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:44.151037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:44.151037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:44.151037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:44.151037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:44.151037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:44.151037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:44.151037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:44.151037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:44.151037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:44.151037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:44.151037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:44.151037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:44.151913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:44.151913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:44.151913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:44.151913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:44.151913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:44.151913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:44.151913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:44.151913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:44.151913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:44.151913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:44.151913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:44.151913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:44.151913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:44.151913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:44.151913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:44.151913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:44.153802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:44.153802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:44.153802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:44.153802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:44.153802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:44.153802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:44.153802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:44.153802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:44.153802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:44.153802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:44.153802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:12:44.153802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:12:44.153802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 82/159] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:12:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:44.158172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4451af3-d24c-49f1-81f1-aa69ff1467ac None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:44.158858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 78/160] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 678 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 19:12:44.164963 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd2f51-2d52-447c-bdb9-4cb63dff769c None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:44.359807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4451af3-d24c-49f1-81f1-aa69ff1467ac None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:44.364404 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd2f51-2d52-447c-bdb9-4cb63dff769c None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:44.381889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:44.422352 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea24d3-215d-47ca-bb0c-1f6434b1baab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:44.422999 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea24d3-215d-47ca-bb0c-1f6434b1baab None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:44.423284 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea24d3-215d-47ca-bb0c-1f6434b1baab None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:44.423559 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea24d3-215d-47ca-bb0c-1f6434b1baab None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:44.424285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea24d3-215d-47ca-bb0c-1f6434b1baab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:44.424591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea24d3-215d-47ca-bb0c-1f6434b1baab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:44.425998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea24d3-215d-47ca-bb0c-1f6434b1baab None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:44.439571 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 79/161] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:12:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:44.440472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea24d3-215d-47ca-bb0c-1f6434b1baab None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:44.444422 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfddb0-afa8-4936-a33c-c176d9145030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:44.445071 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfddb0-afa8-4936-a33c-c176d9145030 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:44.445405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfddb0-afa8-4936-a33c-c176d9145030 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:44.445681 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfddb0-afa8-4936-a33c-c176d9145030 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:44.446436 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfddb0-afa8-4936-a33c-c176d9145030 None admin] RBAC: Adding query filter params (name=member) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:44.446797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfddb0-afa8-4936-a33c-c176d9145030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:44.448246 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfddb0-afa8-4936-a33c-c176d9145030 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:44.468202 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 83/162] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:12:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:44.469160 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfddb0-afa8-4936-a33c-c176d9145030 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:44.586879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea24d3-215d-47ca-bb0c-1f6434b1baab None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:44.630887 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:44.649494 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803f287-9dad-4e86-a39c-76dab673cd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:44.650242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803f287-9dad-4e86-a39c-76dab673cd31 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:44.650362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803f287-9dad-4e86-a39c-76dab673cd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:44.650673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803f287-9dad-4e86-a39c-76dab673cd31 None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:44.656756 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803f287-9dad-4e86-a39c-76dab673cd31 None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:44.658248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803f287-9dad-4e86-a39c-76dab673cd31 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:44.659983 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 80/163] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:44 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:44.660866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803f287-9dad-4e86-a39c-76dab673cd31 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:44.726249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfddb0-afa8-4936-a33c-c176d9145030 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:44.730491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:44.815100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803f287-9dad-4e86-a39c-76dab673cd31 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:44.830068 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc3ee7-ba78-4ecb-a348-84046478cab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:44.836022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc3ee7-ba78-4ecb-a348-84046478cab9 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:44.836022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc3ee7-ba78-4ecb-a348-84046478cab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:44.836022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc3ee7-ba78-4ecb-a348-84046478cab9 None admin] PATH_INFO: `/v3/users/14ef987ec6c24ababb3cce74ff137945` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:44.836677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcc3ee7-ba78-4ecb-a348-84046478cab9 None admin] RBAC: Authorizing `identity:get_user(user_id=14ef987ec6c24ababb3cce74ff137945)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:44.838172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcc3ee7-ba78-4ecb-a348-84046478cab9 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:44.840049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 84/164] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:44 2022] GET /identity/v3/users/14ef987ec6c24ababb3cce74ff137945 => generated 275 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:44.840755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc3ee7-ba78-4ecb-a348-84046478cab9 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:44.900130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a66e2e-1f8e-4fa5-b757-41374161edfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:44.901727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a66e2e-1f8e-4fa5-b757-41374161edfb None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:44.902472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a66e2e-1f8e-4fa5-b757-41374161edfb None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:44.903216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a66e2e-1f8e-4fa5-b757-41374161edfb None admin] PATH_INFO: `/v3/users/14ef987ec6c24ababb3cce74ff137945` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:44.905578 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a66e2e-1f8e-4fa5-b757-41374161edfb None admin] RBAC: Authorizing `identity:get_user(user_id=14ef987ec6c24ababb3cce74ff137945)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:44.908320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a66e2e-1f8e-4fa5-b757-41374161edfb None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:44.911910 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 81/165] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:44 2022] GET /identity/v3/users/14ef987ec6c24ababb3cce74ff137945 => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:44.913583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a66e2e-1f8e-4fa5-b757-41374161edfb None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:45.096010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a66e2e-1f8e-4fa5-b757-41374161edfb None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:45.099253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc3ee7-ba78-4ecb-a348-84046478cab9 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:45.140726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:45.173188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:45.185539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231f81e-ea69-43b8-80a3-093cebd241fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:45.186167 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231f81e-ea69-43b8-80a3-093cebd241fa None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:45.186444 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231f81e-ea69-43b8-80a3-093cebd241fa None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:45.186717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231f81e-ea69-43b8-80a3-093cebd241fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:45.187464 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231f81e-ea69-43b8-80a3-093cebd241fa None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=cabd760392b6495e87b66b85b6a15ca0, user.id=39e61fe22f50447d975033301fd5b706) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:45.187777 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231f81e-ea69-43b8-80a3-093cebd241fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:45.189278 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231f81e-ea69-43b8-80a3-093cebd241fa None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:45.194959 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a48ef-4494-406a-9c9e-1f16eb9720b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:45.195589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a48ef-4494-406a-9c9e-1f16eb9720b3 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:45.195868 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a48ef-4494-406a-9c9e-1f16eb9720b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:45.196169 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a48ef-4494-406a-9c9e-1f16eb9720b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:45.209821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 85/166] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:12:44 2022] GET /identity/v3/role_assignments?user.id=39e61fe22f50447d975033301fd5b706&scope.project.id=cabd760392b6495e87b66b85b6a15ca0&role.id=4da42f08d107448fb877b42b61bc74ea => generated 608 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:45.216469 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393da66-00a0-4f04-8bcd-b8777b2c91c1 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:45.216755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393da66-00a0-4f04-8bcd-b8777b2c91c1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:45.217019 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393da66-00a0-4f04-8bcd-b8777b2c91c1 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:45.218570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 86/167] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:45.220650 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c393da66-00a0-4f04-8bcd-b8777b2c91c1 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:45.389209 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a48ef-4494-406a-9c9e-1f16eb9720b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:45.390731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a48ef-4494-406a-9c9e-1f16eb9720b3 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:45.513978 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c393da66-00a0-4f04-8bcd-b8777b2c91c1 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:45.603950 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0399ea-6ecc-4a70-9764-020f0b924559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:45.604611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0399ea-6ecc-4a70-9764-020f0b924559 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:45.604904 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0399ea-6ecc-4a70-9764-020f0b924559 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:45.605157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0399ea-6ecc-4a70-9764-020f0b924559 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:45.647858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 82/168] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 735 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:12:45.650851 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c73de8-54f8-4bdb-9838-58c0e542a223 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:45.651384 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c73de8-54f8-4bdb-9838-58c0e542a223 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:45.651645 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c73de8-54f8-4bdb-9838-58c0e542a223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:45.693319 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-09c73de8-54f8-4bdb-9838-58c0e542a223 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:45.697123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-09c73de8-54f8-4bdb-9838-58c0e542a223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:45.716542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:45.757394 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:45.777677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0399ea-6ecc-4a70-9764-020f0b924559 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:45.779198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0399ea-6ecc-4a70-9764-020f0b924559 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:45.805919 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 83/169] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:45.807147 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c73de8-54f8-4bdb-9838-58c0e542a223 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:45.943299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 87/170] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 724 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:12:45.951886 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58739e-1322-470f-9280-668bf484438b None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:45.951886 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58739e-1322-470f-9280-668bf484438b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:45.951886 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58739e-1322-470f-9280-668bf484438b None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:45.951886 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 88/171] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:45.951886 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198e9ee-f2ff-4220-87ee-93325eebd7f4 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:45.951886 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198e9ee-f2ff-4220-87ee-93325eebd7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:45.952495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5198e9ee-f2ff-4220-87ee-93325eebd7f4 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:45.952495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 89/172] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:45.953372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198e9ee-f2ff-4220-87ee-93325eebd7f4 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:45.971468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c73de8-54f8-4bdb-9838-58c0e542a223 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:46.017599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf3a5c-97cf-4442-b95c-1e48aa9fa6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:46.018266 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf3a5c-97cf-4442-b95c-1e48aa9fa6ee None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:46.018628 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf3a5c-97cf-4442-b95c-1e48aa9fa6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:46.018807 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf3a5c-97cf-4442-b95c-1e48aa9fa6ee None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:46.020012 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf3a5c-97cf-4442-b95c-1e48aa9fa6ee None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:46.021637 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf3a5c-97cf-4442-b95c-1e48aa9fa6ee None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:46.023605 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 84/173] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:45 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:46.024834 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef721b-8438-4af4-8ecf-812749f86828 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:46.025089 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef721b-8438-4af4-8ecf-812749f86828 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:46.025353 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef721b-8438-4af4-8ecf-812749f86828 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:46.030630 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 85/174] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:46.030630 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca04766-2bcf-464e-88d3-c15d8f5b409b None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:46.030630 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca04766-2bcf-464e-88d3-c15d8f5b409b None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:46.030630 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca04766-2bcf-464e-88d3-c15d8f5b409b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:46.046739 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-2ca04766-2bcf-464e-88d3-c15d8f5b409b None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:46.048089 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-2ca04766-2bcf-464e-88d3-c15d8f5b409b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:46.058381 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:46.072629 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:46.081485 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5198e9ee-f2ff-4220-87ee-93325eebd7f4 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:46.117938 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57c9671-e6bd-48b3-94a5-1469d2a23d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:46.118563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c9671-e6bd-48b3-94a5-1469d2a23d22 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:46.118836 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c9671-e6bd-48b3-94a5-1469d2a23d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:46.119105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57c9671-e6bd-48b3-94a5-1469d2a23d22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:46.132088 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 86/175] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:46.132088 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca04766-2bcf-464e-88d3-c15d8f5b409b None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:46.135315 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57c9671-e6bd-48b3-94a5-1469d2a23d22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:46.140463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-e57c9671-e6bd-48b3-94a5-1469d2a23d22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:46.140463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:46.140463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:46.140463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:46.140463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:46.140463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:46.140463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:46.140463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:46.140463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:46.140463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:46.140463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:46.140463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:46.140463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:46.140463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:46.140463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:46.140463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:46.141298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:46.141298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:46.141298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:46.141298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:46.141298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:46.141298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:46.141298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:46.141298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:46.141298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:46.141298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:46.141298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:46.141298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:46.141298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:46.141298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:46.141298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:46.141298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:46.142092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:46.142092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:46.142092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:46.142092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:46.142092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:46.142092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:46.142092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:46.142092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:46.142092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:46.142092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:46.142092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:46.142092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:12:46.142684 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 90/176] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:46.147276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9ec1d-fac5-4f35-9d49-ea261fd47915 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:46.147276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9ec1d-fac5-4f35-9d49-ea261fd47915 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:46.147276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9ec1d-fac5-4f35-9d49-ea261fd47915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:46.163657 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-0aa9ec1d-fac5-4f35-9d49-ea261fd47915 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:46.164870 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0aa9ec1d-fac5-4f35-9d49-ea261fd47915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:46.233026 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 91/177] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:46.233879 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9ec1d-fac5-4f35-9d49-ea261fd47915 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:46.244438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca04766-2bcf-464e-88d3-c15d8f5b409b None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:46.280878 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f3c37-d8f0-4502-aa7d-1e1c58e8ab5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:46.281387 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f3c37-d8f0-4502-aa7d-1e1c58e8ab5e None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:46.281658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f3c37-d8f0-4502-aa7d-1e1c58e8ab5e None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:46.281926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f3c37-d8f0-4502-aa7d-1e1c58e8ab5e None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:46.283081 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f3c37-d8f0-4502-aa7d-1e1c58e8ab5e None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:46.284540 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f3c37-d8f0-4502-aa7d-1e1c58e8ab5e None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:46.286355 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 87/178] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:46 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:46.287319 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb002d5-c622-4567-8a8d-f5d225b6d150 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:46.287596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb002d5-c622-4567-8a8d-f5d225b6d150 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:46.287864 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb002d5-c622-4567-8a8d-f5d225b6d150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:46.305621 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-6bb002d5-c622-4567-8a8d-f5d225b6d150 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:46.306805 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6bb002d5-c622-4567-8a8d-f5d225b6d150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:46.310044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:46.326986 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:46.344158 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9ec1d-fac5-4f35-9d49-ea261fd47915 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:46.374209 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 88/179] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:46.375067 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb002d5-c622-4567-8a8d-f5d225b6d150 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:46.378410 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f15b27-32d1-4487-9ea0-6817136f3ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:46.379072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f15b27-32d1-4487-9ea0-6817136f3ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:46.379389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f15b27-32d1-4487-9ea0-6817136f3ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:46.379686 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f15b27-32d1-4487-9ea0-6817136f3ad5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:46.380499 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f15b27-32d1-4487-9ea0-6817136f3ad5 None admin] RBAC: Adding query filter params (role.id=9a1c954216e34607a6788f44f07a593b, scope.project.id=cabd760392b6495e87b66b85b6a15ca0, user.id=14ef987ec6c24ababb3cce74ff137945) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:46.380805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f15b27-32d1-4487-9ea0-6817136f3ad5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:46.382263 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f15b27-32d1-4487-9ea0-6817136f3ad5 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:46.389351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 92/180] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:12:46 2022] GET /identity/v3/role_assignments?user.id=14ef987ec6c24ababb3cce74ff137945&scope.project.id=cabd760392b6495e87b66b85b6a15ca0&role.id=9a1c954216e34607a6788f44f07a593b => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:46.390209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f15b27-32d1-4487-9ea0-6817136f3ad5 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:46.507221 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb002d5-c622-4567-8a8d-f5d225b6d150 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:46.514740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f15b27-32d1-4487-9ea0-6817136f3ad5 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:46.547509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf27062-a01e-4976-a518-745956486df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:46.548134 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf27062-a01e-4976-a518-745956486df2 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:46.548496 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf27062-a01e-4976-a518-745956486df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:46.549346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf27062-a01e-4976-a518-745956486df2 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:46.549435 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf27062-a01e-4976-a518-745956486df2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:46.549701 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf27062-a01e-4976-a518-745956486df2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:46.551671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf27062-a01e-4976-a518-745956486df2 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:46.553006 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba68d6-1f28-4bff-b467-4c9dbdd2814a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:46.553805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba68d6-1f28-4bff-b467-4c9dbdd2814a None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:46.553921 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba68d6-1f28-4bff-b467-4c9dbdd2814a None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:46.554189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba68d6-1f28-4bff-b467-4c9dbdd2814a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:46.562997 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 89/181] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:12:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:46.565189 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc9537-615f-4ad7-bf10-e63fae3c6fd4 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:46.565656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc9537-615f-4ad7-bf10-e63fae3c6fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:46.566101 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc9537-615f-4ad7-bf10-e63fae3c6fd4 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:46.567570 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 90/182] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:46.568250 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba68d6-1f28-4bff-b467-4c9dbdd2814a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:46.568803 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc9537-615f-4ad7-bf10-e63fae3c6fd4 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:46.572657 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:46.574622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-57ba68d6-1f28-4bff-b467-4c9dbdd2814a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:46.574622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:46.574622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:46.574622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:46.574622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:46.574622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:46.574622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:46.574622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:46.574622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:46.574622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:46.574622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:46.574622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:46.574622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:46.574622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:46.574622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:46.574622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:46.575511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:46.575511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:46.575511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:46.575511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:46.575511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:46.575511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:46.575511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:46.575511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:46.575511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:46.575511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:46.575511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:46.575511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:46.575511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:46.575511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:46.575511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:46.575511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:46.576369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:46.576369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:46.576369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:46.576369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:46.576369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:46.576369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:46.576369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:46.576369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:46.576369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:46.576369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:46.576369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:46.576369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:12:46.576369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 93/183] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:12:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:46.577051 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba68d6-1f28-4bff-b467-4c9dbdd2814a None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:46.592234 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:46.687789 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc9537-615f-4ad7-bf10-e63fae3c6fd4 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:46.710168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba68d6-1f28-4bff-b467-4c9dbdd2814a None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:46.722486 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee854d-dbc6-446f-868a-c9e1c220497c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:46.723125 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee854d-dbc6-446f-868a-c9e1c220497c None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:46.723403 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee854d-dbc6-446f-868a-c9e1c220497c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:46.723675 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee854d-dbc6-446f-868a-c9e1c220497c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:46.724805 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee854d-dbc6-446f-868a-c9e1c220497c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:46.726247 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee854d-dbc6-446f-868a-c9e1c220497c None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:46.727805 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 91/184] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:12:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:46.728605 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee854d-dbc6-446f-868a-c9e1c220497c None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:46.747010 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde10e6c-6106-4b31-b707-13a13a673fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:46.747644 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde10e6c-6106-4b31-b707-13a13a673fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:46.747931 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde10e6c-6106-4b31-b707-13a13a673fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:46.748262 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde10e6c-6106-4b31-b707-13a13a673fe7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:46.748990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde10e6c-6106-4b31-b707-13a13a673fe7 None admin] RBAC: Adding query filter params (role.id=246bbd6878334bba893bc6f5bf5caee0, scope.project.id=cabd760392b6495e87b66b85b6a15ca0, user.id=14ef987ec6c24ababb3cce74ff137945) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:46.749556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde10e6c-6106-4b31-b707-13a13a673fe7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:46.751057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde10e6c-6106-4b31-b707-13a13a673fe7 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:46.757232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 94/185] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:12:46 2022] GET /identity/v3/role_assignments?user.id=14ef987ec6c24ababb3cce74ff137945&scope.project.id=cabd760392b6495e87b66b85b6a15ca0&role.id=246bbd6878334bba893bc6f5bf5caee0 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:46.758016 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde10e6c-6106-4b31-b707-13a13a673fe7 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:46.830689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:46.851256 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:46.875174 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee854d-dbc6-446f-868a-c9e1c220497c None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:46.887760 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde10e6c-6106-4b31-b707-13a13a673fe7 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:46.918727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb941e-3277-4790-a632-61885cd0cd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:46.919470 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb941e-3277-4790-a632-61885cd0cd1b None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:46.919780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb941e-3277-4790-a632-61885cd0cd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:46.920064 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb941e-3277-4790-a632-61885cd0cd1b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:46.928043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4670829-8f2d-4bba-8647-b141f7d2393f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:46.928697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4670829-8f2d-4bba-8647-b141f7d2393f None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:46.928980 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4670829-8f2d-4bba-8647-b141f7d2393f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:46.929284 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4670829-8f2d-4bba-8647-b141f7d2393f None admin] PATH_INFO: `/v3/users/14ef987ec6c24ababb3cce74ff137945` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:46.930839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4670829-8f2d-4bba-8647-b141f7d2393f None admin] RBAC: Authorizing `identity:get_user(user_id=14ef987ec6c24ababb3cce74ff137945)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:46.932354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4670829-8f2d-4bba-8647-b141f7d2393f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:46.933664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb941e-3277-4790-a632-61885cd0cd1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:46.935065 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 95/186] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:46 2022] GET /identity/v3/users/14ef987ec6c24ababb3cce74ff137945 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:46.935552 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3dcbaa-f951-4e06-aac9-8b5d7ccd7a1a None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:46.935833 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3dcbaa-f951-4e06-aac9-8b5d7ccd7a1a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:46.937476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3dcbaa-f951-4e06-aac9-8b5d7ccd7a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:46.940100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-a3cb941e-3277-4790-a632-61885cd0cd1b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:12:46.940100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:46.940100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:46.940100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:46.940100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:46.940100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:46.940100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:46.940100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:46.940100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:46.940100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:46.940100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:46.940100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:46.940100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:46.940100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:46.940100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:46.940100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:46.941037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:46.941037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:46.941037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:46.941037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:46.941037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:46.941037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:46.941037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:46.941037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:46.941037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:46.941037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:46.941037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:46.941037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:46.941037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:46.941037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:46.941037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:46.941037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:46.941953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:46.941953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:46.941953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:46.941953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:46.941953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:46.941953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:46.941953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:46.941953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:46.941953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:46.941953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:46.941953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:12:46.941953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:46.941953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 92/187] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:12:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:46.942667 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb941e-3277-4790-a632-61885cd0cd1b None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:46.961261 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-3b3dcbaa-f951-4e06-aac9-8b5d7ccd7a1a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:46.962807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3b3dcbaa-f951-4e06-aac9-8b5d7ccd7a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:47.049302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 96/188] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:47.051080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3dcbaa-f951-4e06-aac9-8b5d7ccd7a1a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:47.055325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb941e-3277-4790-a632-61885cd0cd1b None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:47.086602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:47.093437 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592467d3-0bb9-4be2-9fd8-abee7030cb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:47.094191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592467d3-0bb9-4be2-9fd8-abee7030cb9b None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:47.094596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592467d3-0bb9-4be2-9fd8-abee7030cb9b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:47.094955 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592467d3-0bb9-4be2-9fd8-abee7030cb9b None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:47.095717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592467d3-0bb9-4be2-9fd8-abee7030cb9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:47.096158 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592467d3-0bb9-4be2-9fd8-abee7030cb9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:47.097741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592467d3-0bb9-4be2-9fd8-abee7030cb9b None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:47.106479 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 93/189] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:47.107296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592467d3-0bb9-4be2-9fd8-abee7030cb9b None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:47.135055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:47.166556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3dcbaa-f951-4e06-aac9-8b5d7ccd7a1a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:47.201631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401af6e-2e64-4b3c-be6e-6f2bf0b69476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:47.202553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401af6e-2e64-4b3c-be6e-6f2bf0b69476 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:47.202976 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401af6e-2e64-4b3c-be6e-6f2bf0b69476 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:47.203253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401af6e-2e64-4b3c-be6e-6f2bf0b69476 None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:47.203903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d401af6e-2e64-4b3c-be6e-6f2bf0b69476 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:47.205405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d401af6e-2e64-4b3c-be6e-6f2bf0b69476 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:47.210596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592467d3-0bb9-4be2-9fd8-abee7030cb9b None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:47.226424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 97/190] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:12:47 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:12:47.227438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401af6e-2e64-4b3c-be6e-6f2bf0b69476 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:47.261585 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbfb10-5044-4679-80e7-106bdd873cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:47.262293 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbfb10-5044-4679-80e7-106bdd873cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:47.262621 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbfb10-5044-4679-80e7-106bdd873cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:47.262850 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbfb10-5044-4679-80e7-106bdd873cf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:47.337912 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:47.350546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbfb10-5044-4679-80e7-106bdd873cf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:47.352112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bbfb10-5044-4679-80e7-106bdd873cf9 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:47.353816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401af6e-2e64-4b3c-be6e-6f2bf0b69476 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:47.383619 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:47.400664 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745b812-0c5a-441e-904f-158019d51aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:47.401380 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745b812-0c5a-441e-904f-158019d51aec None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:47.401723 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745b812-0c5a-441e-904f-158019d51aec None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:47.402059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2745b812-0c5a-441e-904f-158019d51aec None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:47.402942 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745b812-0c5a-441e-904f-158019d51aec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:47.403239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745b812-0c5a-441e-904f-158019d51aec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:47.404662 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2745b812-0c5a-441e-904f-158019d51aec None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:47.413907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 98/191] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:12:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:47.414715 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745b812-0c5a-441e-904f-158019d51aec None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:47.472944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 94/192] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:12:47.473829 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbfb10-5044-4679-80e7-106bdd873cf9 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:47.531516 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2745b812-0c5a-441e-904f-158019d51aec None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:47.569745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df896ca5-2dd1-4bd2-9066-a0206dae38ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:47.570389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df896ca5-2dd1-4bd2-9066-a0206dae38ae None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:47.570668 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df896ca5-2dd1-4bd2-9066-a0206dae38ae None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:47.570939 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df896ca5-2dd1-4bd2-9066-a0206dae38ae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:47.581734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df896ca5-2dd1-4bd2-9066-a0206dae38ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:47.586877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-df896ca5-2dd1-4bd2-9066-a0206dae38ae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:12:47.586877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:47.586877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:47.586877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:47.586877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:47.586877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:47.586877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:47.586877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:47.586877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:47.586877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:47.586877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:47.586877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:47.586877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:47.586877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:47.586877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:47.586877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:47.587767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:47.587767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:47.587767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:47.587767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:47.587767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:47.587767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:47.587767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:47.587767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:47.587767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:47.587767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:47.587767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:47.587767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:47.587767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:47.587767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:47.587767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:47.587767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:47.588656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:47.588656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:47.588656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:47.588656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:47.588656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:47.588656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:47.588656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:47.588656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:47.588656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:47.588656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:47.588656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:12:47.588656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:12:47.588656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 99/193] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:47.589394 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df896ca5-2dd1-4bd2-9066-a0206dae38ae None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:47.606072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:47.618734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bbfb10-5044-4679-80e7-106bdd873cf9 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:47.642310 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:47.669304 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6871a-22f5-47bb-8c29-ef5e321d4cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:47.670580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6871a-22f5-47bb-8c29-ef5e321d4cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:47.671211 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6871a-22f5-47bb-8c29-ef5e321d4cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:47.671908 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6871a-22f5-47bb-8c29-ef5e321d4cd8 None admin] PATH_INFO: `/v3/users/67cbfba3427c42ba9f403be29873efd9` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:47.693846 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6871a-22f5-47bb-8c29-ef5e321d4cd8 None admin] RBAC: Authorizing `identity:get_user(user_id=67cbfba3427c42ba9f403be29873efd9)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:47.695171 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df896ca5-2dd1-4bd2-9066-a0206dae38ae None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:47.697157 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6871a-22f5-47bb-8c29-ef5e321d4cd8 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:47.699723 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 95/194] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:47 2022] GET /identity/v3/users/67cbfba3427c42ba9f403be29873efd9 => generated 283 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:47.701133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6871a-22f5-47bb-8c29-ef5e321d4cd8 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:47.730080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106608bb-859b-4940-a4dc-8bfb5bc6afca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:47.730717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106608bb-859b-4940-a4dc-8bfb5bc6afca None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:47.731064 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106608bb-859b-4940-a4dc-8bfb5bc6afca None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:47.731333 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106608bb-859b-4940-a4dc-8bfb5bc6afca None admin] PATH_INFO: `/v3/users/67cbfba3427c42ba9f403be29873efd9` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:47.732532 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106608bb-859b-4940-a4dc-8bfb5bc6afca None admin] RBAC: Authorizing `identity:get_user(user_id=67cbfba3427c42ba9f403be29873efd9)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:47.734423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106608bb-859b-4940-a4dc-8bfb5bc6afca None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:47.735916 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 100/195] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:47 2022] GET /identity/v3/users/67cbfba3427c42ba9f403be29873efd9 => generated 283 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:47.736831 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106608bb-859b-4940-a4dc-8bfb5bc6afca None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:47.842548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106608bb-859b-4940-a4dc-8bfb5bc6afca None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:47.854419 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6871a-22f5-47bb-8c29-ef5e321d4cd8 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:47.860714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:47.891254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc8fec-6f53-4d75-9cbf-b5effd3faf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:47.893431 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc8fec-6f53-4d75-9cbf-b5effd3faf3e None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:47.894412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc8fec-6f53-4d75-9cbf-b5effd3faf3e None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:47.894918 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc8fec-6f53-4d75-9cbf-b5effd3faf3e None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:47.896449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc8fec-6f53-4d75-9cbf-b5effd3faf3e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:47.897112 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc8fec-6f53-4d75-9cbf-b5effd3faf3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:47.898878 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc8fec-6f53-4d75-9cbf-b5effd3faf3e None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:47.901316 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f39c19-86dd-4769-b9d7-3513f239b97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:47.901956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f39c19-86dd-4769-b9d7-3513f239b97b None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:47.902251 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f39c19-86dd-4769-b9d7-3513f239b97b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:47.902519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f39c19-86dd-4769-b9d7-3513f239b97b None admin] PATH_INFO: `/v3/projects/edcd2e6e9a8f4c3ebec542d289cb0c95` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:47.903681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f39c19-86dd-4769-b9d7-3513f239b97b None admin] RBAC: Authorizing `identity:get_project(project_id=edcd2e6e9a8f4c3ebec542d289cb0c95)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:47.905126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f39c19-86dd-4769-b9d7-3513f239b97b None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:47.910629 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 101/196] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:12:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:47.911764 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 96/197] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:47 2022] GET /identity/v3/projects/edcd2e6e9a8f4c3ebec542d289cb0c95 => generated 312 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:47.912208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc8fec-6f53-4d75-9cbf-b5effd3faf3e None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:47.912662 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f39c19-86dd-4769-b9d7-3513f239b97b None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:47.933992 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:48.041891 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f39c19-86dd-4769-b9d7-3513f239b97b None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:48.054321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc8fec-6f53-4d75-9cbf-b5effd3faf3e None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:48.079381 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a41dd-eec8-46ea-9027-02626f12665b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:48.080265 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a41dd-eec8-46ea-9027-02626f12665b None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:48.080677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a41dd-eec8-46ea-9027-02626f12665b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:48.081208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a41dd-eec8-46ea-9027-02626f12665b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:48.108283 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536fca82-8c26-44a5-8222-a32fb0ec5ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:48.108990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fca82-8c26-44a5-8222-a32fb0ec5ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:48.109365 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fca82-8c26-44a5-8222-a32fb0ec5ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:48.109740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536fca82-8c26-44a5-8222-a32fb0ec5ac3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:48.156234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a41dd-eec8-46ea-9027-02626f12665b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:48.157980 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a41dd-eec8-46ea-9027-02626f12665b None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:48.158405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:48.177950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:48.182376 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536fca82-8c26-44a5-8222-a32fb0ec5ac3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:48.184387 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536fca82-8c26-44a5-8222-a32fb0ec5ac3 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:48.266592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 97/198] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:12:48.267759 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a41dd-eec8-46ea-9027-02626f12665b None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:48.286560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 102/199] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:12:48.287452 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536fca82-8c26-44a5-8222-a32fb0ec5ac3 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:48.367970 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:48.369394 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a41dd-eec8-46ea-9027-02626f12665b None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:48.388309 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:48.392720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536fca82-8c26-44a5-8222-a32fb0ec5ac3 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:48.407694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f6991-3dc9-46c1-81ee-e6b89bfde367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:48.408559 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f6991-3dc9-46c1-81ee-e6b89bfde367 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:48.408970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f6991-3dc9-46c1-81ee-e6b89bfde367 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:48.409371 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f6991-3dc9-46c1-81ee-e6b89bfde367 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:48.410246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f6991-3dc9-46c1-81ee-e6b89bfde367 None admin] RBAC: Adding query filter params (role.id=246bbd6878334bba893bc6f5bf5caee0, scope.project.id=edcd2e6e9a8f4c3ebec542d289cb0c95, user.id=14ef987ec6c24ababb3cce74ff137945) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:48.410685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f6991-3dc9-46c1-81ee-e6b89bfde367 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:48.412290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f6991-3dc9-46c1-81ee-e6b89bfde367 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:48.418624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 98/200] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:12:48 2022] GET /identity/v3/role_assignments?user.id=14ef987ec6c24ababb3cce74ff137945&scope.project.id=edcd2e6e9a8f4c3ebec542d289cb0c95&role.id=246bbd6878334bba893bc6f5bf5caee0 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:48.425172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f6991-3dc9-46c1-81ee-e6b89bfde367 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:48.430416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b446779-7b96-40aa-8a2e-f9c614506f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:48.430570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b446779-7b96-40aa-8a2e-f9c614506f4b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:48.430955 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b446779-7b96-40aa-8a2e-f9c614506f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:48.431321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b446779-7b96-40aa-8a2e-f9c614506f4b None admin] PATH_INFO: `/v3/users/29d18eba49914276818391057165bf4f` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:48.432619 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b446779-7b96-40aa-8a2e-f9c614506f4b None admin] RBAC: Authorizing `identity:get_user(user_id=29d18eba49914276818391057165bf4f)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:48.440018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b446779-7b96-40aa-8a2e-f9c614506f4b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:48.440018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 103/201] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:48 2022] GET /identity/v3/users/29d18eba49914276818391057165bf4f => generated 289 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:48.440018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b446779-7b96-40aa-8a2e-f9c614506f4b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:48.578300 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b446779-7b96-40aa-8a2e-f9c614506f4b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:48.631601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f6991-3dc9-46c1-81ee-e6b89bfde367 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:48.657649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef096d-c4d8-485e-ab14-8810ec1a1a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:48.659254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef096d-c4d8-485e-ab14-8810ec1a1a29 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:48.659536 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef096d-c4d8-485e-ab14-8810ec1a1a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:48.659812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef096d-c4d8-485e-ab14-8810ec1a1a29 None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:48.661151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef096d-c4d8-485e-ab14-8810ec1a1a29 None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:48.664283 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef096d-c4d8-485e-ab14-8810ec1a1a29 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:48.666017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 104/202] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:48 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:48.668866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef096d-c4d8-485e-ab14-8810ec1a1a29 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:48.692292 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:48.705937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b044c-17bf-493a-aeef-1c2af512c054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:48.706583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b044c-17bf-493a-aeef-1c2af512c054 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:48.706866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b044c-17bf-493a-aeef-1c2af512c054 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:48.707141 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b044c-17bf-493a-aeef-1c2af512c054 None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:48.709507 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b044c-17bf-493a-aeef-1c2af512c054 None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:48.712268 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80b044c-17bf-493a-aeef-1c2af512c054 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:48.712268 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 99/203] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:48 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:48.714241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b044c-17bf-493a-aeef-1c2af512c054 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:48.786238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:48.819927 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef096d-c4d8-485e-ab14-8810ec1a1a29 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:48.844782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80b044c-17bf-493a-aeef-1c2af512c054 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:48.856275 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8adaf9-2508-4f2f-ab4a-db6a33dd7ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:48.856949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8adaf9-2508-4f2f-ab4a-db6a33dd7ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:48.857178 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8adaf9-2508-4f2f-ab4a-db6a33dd7ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:48.857453 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8adaf9-2508-4f2f-ab4a-db6a33dd7ba4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:48.885996 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa44552-a17c-4c16-9776-fc0b7fc78ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:48.887003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa44552-a17c-4c16-9776-fc0b7fc78ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:48.887648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa44552-a17c-4c16-9776-fc0b7fc78ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:48.888149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa44552-a17c-4c16-9776-fc0b7fc78ef0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:48.889212 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa44552-a17c-4c16-9776-fc0b7fc78ef0 None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=ad772dbadd1e498f9d58bf65502c4469, user.id=67cbfba3427c42ba9f403be29873efd9) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:48.889761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa44552-a17c-4c16-9776-fc0b7fc78ef0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:48.891432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa44552-a17c-4c16-9776-fc0b7fc78ef0 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:48.898518 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 100/204] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:12:48 2022] GET /identity/v3/role_assignments?user.id=67cbfba3427c42ba9f403be29873efd9&scope.project.id=ad772dbadd1e498f9d58bf65502c4469&role.id=4da42f08d107448fb877b42b61bc74ea => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:48.900732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa44552-a17c-4c16-9776-fc0b7fc78ef0 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:48.980049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8adaf9-2508-4f2f-ab4a-db6a33dd7ba4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:48.981812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8adaf9-2508-4f2f-ab4a-db6a33dd7ba4 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:49.029493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:49.039474 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa44552-a17c-4c16-9776-fc0b7fc78ef0 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:49.066405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:49.082057 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f078a2-3987-4a0a-9de8-0fe094d40dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:49.082868 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f078a2-3987-4a0a-9de8-0fe094d40dcc None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:49.083255 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f078a2-3987-4a0a-9de8-0fe094d40dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:49.083636 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f078a2-3987-4a0a-9de8-0fe094d40dcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:49.084606 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f078a2-3987-4a0a-9de8-0fe094d40dcc None admin] RBAC: Adding query filter params (role.id=9a1c954216e34607a6788f44f07a593b, scope.project.id=ad772dbadd1e498f9d58bf65502c4469, user.id=67cbfba3427c42ba9f403be29873efd9) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:49.085026 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f078a2-3987-4a0a-9de8-0fe094d40dcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:49.086679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f078a2-3987-4a0a-9de8-0fe094d40dcc None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:49.093537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 101/205] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:12:48 2022] GET /identity/v3/role_assignments?user.id=67cbfba3427c42ba9f403be29873efd9&scope.project.id=ad772dbadd1e498f9d58bf65502c4469&role.id=9a1c954216e34607a6788f44f07a593b => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:49.216143 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 105/206] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:12:49.222193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f078a2-3987-4a0a-9de8-0fe094d40dcc None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:49.335819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f078a2-3987-4a0a-9de8-0fe094d40dcc None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:49.373420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac5767-3e77-49a5-af8b-0762ec0322cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:49.374222 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac5767-3e77-49a5-af8b-0762ec0322cf None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:49.374608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac5767-3e77-49a5-af8b-0762ec0322cf None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:49.375134 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac5767-3e77-49a5-af8b-0762ec0322cf None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:49.376462 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac5767-3e77-49a5-af8b-0762ec0322cf None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:49.378033 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac5767-3e77-49a5-af8b-0762ec0322cf None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:49.382482 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 102/207] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:49 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:49.386458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8adaf9-2508-4f2f-ab4a-db6a33dd7ba4 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:49.464420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221531e0-5930-45e9-b610-c01c0f2b4646 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:49.464983 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221531e0-5930-45e9-b610-c01c0f2b4646 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:49.465410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221531e0-5930-45e9-b610-c01c0f2b4646 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:49.467002 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 103/208] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:12:49.472540 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd310cdf-36af-4be5-8019-768c2483a65d None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:49.472822 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd310cdf-36af-4be5-8019-768c2483a65d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:49.473095 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd310cdf-36af-4be5-8019-768c2483a65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:49.499368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-fd310cdf-36af-4be5-8019-768c2483a65d None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:49.500597 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-fd310cdf-36af-4be5-8019-768c2483a65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:49.538080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8adaf9-2508-4f2f-ab4a-db6a33dd7ba4 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:49.566562 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:49.577338 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:49.601756 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 104/209] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:49.608667 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5004e9-11d8-47bb-a462-b6c3651ea069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:49.609292 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5004e9-11d8-47bb-a462-b6c3651ea069 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:49.609576 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5004e9-11d8-47bb-a462-b6c3651ea069 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:49.609844 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5004e9-11d8-47bb-a462-b6c3651ea069 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:49.610576 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5004e9-11d8-47bb-a462-b6c3651ea069 None admin] RBAC: Adding query filter params (role.id=31d708bdce024bbc8423f085d27b1776, scope.project.id=cabd760392b6495e87b66b85b6a15ca0, user.id=29d18eba49914276818391057165bf4f) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:49.610881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5004e9-11d8-47bb-a462-b6c3651ea069 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:49.612373 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5004e9-11d8-47bb-a462-b6c3651ea069 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:49.619078 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 106/210] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:12:49 2022] GET /identity/v3/role_assignments?user.id=29d18eba49914276818391057165bf4f&scope.project.id=cabd760392b6495e87b66b85b6a15ca0&role.id=31d708bdce024bbc8423f085d27b1776 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:49.645434 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd310cdf-36af-4be5-8019-768c2483a65d None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:49.700990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e7090-b072-4e43-9a20-501211d9052c None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:49.701264 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e7090-b072-4e43-9a20-501211d9052c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:49.701540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e7090-b072-4e43-9a20-501211d9052c None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:49.703181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 107/211] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:49.708920 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05067fb-8f24-4fb8-8bcb-7875467c1208 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:49.709197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05067fb-8f24-4fb8-8bcb-7875467c1208 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:49.709476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05067fb-8f24-4fb8-8bcb-7875467c1208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:49.757219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-d05067fb-8f24-4fb8-8bcb-7875467c1208 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:49.758351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-d05067fb-8f24-4fb8-8bcb-7875467c1208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:49.783412 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd310cdf-36af-4be5-8019-768c2483a65d None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:49.821041 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3a949-f09d-4d69-96e8-50ecb1b88f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:49.821951 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3a949-f09d-4d69-96e8-50ecb1b88f3c None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:49.822409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3a949-f09d-4d69-96e8-50ecb1b88f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:49.822786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3a949-f09d-4d69-96e8-50ecb1b88f3c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:49.834682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3a949-f09d-4d69-96e8-50ecb1b88f3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:49.840129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-d0f3a949-f09d-4d69-96e8-50ecb1b88f3c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:49.840129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:49.840129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:49.840129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:49.840129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:49.840129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:49.840129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:49.840129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:49.840129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:49.840129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:49.840129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:49.840129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:49.840129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:49.840129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:49.840129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:49.840129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:49.841230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:49.841230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:49.841230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:49.841230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:49.841230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:49.841230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:49.841230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:49.841230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:49.841230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:49.841230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:49.841230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:49.841230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:49.841230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:49.841230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:49.841230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:49.841230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:49.842121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:49.842121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:49.842121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:49.842121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:49.842121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:49.842121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:49.842121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:49.842121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:49.842121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:49.842121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:49.842121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:49.842121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:49.842121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 105/212] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:49.848087 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 108/213] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:12:49.861995 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3a949-f09d-4d69-96e8-50ecb1b88f3c None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:49.872662 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05067fb-8f24-4fb8-8bcb-7875467c1208 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:49.915319 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:50.004977 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05067fb-8f24-4fb8-8bcb-7875467c1208 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:50.007520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3a949-f09d-4d69-96e8-50ecb1b88f3c None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:50.026879 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:50.056504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664decc-fc29-492d-b21c-9f5918abda0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:50.057809 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664decc-fc29-492d-b21c-9f5918abda0d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:50.057809 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664decc-fc29-492d-b21c-9f5918abda0d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:50.057809 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664decc-fc29-492d-b21c-9f5918abda0d None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:50.059252 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664decc-fc29-492d-b21c-9f5918abda0d None admin] RBAC: Adding query filter params (name=member) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:50.059359 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664decc-fc29-492d-b21c-9f5918abda0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:50.061761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664decc-fc29-492d-b21c-9f5918abda0d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:50.067266 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad033ae7-42cd-4fb8-bf28-5028414dcf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:50.068236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad033ae7-42cd-4fb8-bf28-5028414dcf8f None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:50.068940 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad033ae7-42cd-4fb8-bf28-5028414dcf8f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:50.069379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad033ae7-42cd-4fb8-bf28-5028414dcf8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:50.070729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 106/214] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:12:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:50.071543 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad033ae7-42cd-4fb8-bf28-5028414dcf8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:50.073282 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad033ae7-42cd-4fb8-bf28-5028414dcf8f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:50.075666 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 109/215] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:12:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:50.079266 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664decc-fc29-492d-b21c-9f5918abda0d None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:50.082863 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad033ae7-42cd-4fb8-bf28-5028414dcf8f None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:50.209337 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:50.211825 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad033ae7-42cd-4fb8-bf28-5028414dcf8f None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:50.216361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664decc-fc29-492d-b21c-9f5918abda0d None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:50.255448 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b404c-842a-445f-a2b1-e6892e9f8aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:50.256129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b404c-842a-445f-a2b1-e6892e9f8aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:50.256429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b404c-842a-445f-a2b1-e6892e9f8aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:50.256716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b404c-842a-445f-a2b1-e6892e9f8aa7 None admin] PATH_INFO: `/v3/users/07d34fb8652e423dbe23f77fa9277e1b` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:50.258012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a54ce9-02d0-4e5d-82e5-e02c58625674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:50.259091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a54ce9-02d0-4e5d-82e5-e02c58625674 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:50.260377 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a54ce9-02d0-4e5d-82e5-e02c58625674 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:50.260482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a54ce9-02d0-4e5d-82e5-e02c58625674 None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:50.261149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a54ce9-02d0-4e5d-82e5-e02c58625674 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:50.262697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a54ce9-02d0-4e5d-82e5-e02c58625674 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:50.277198 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b404c-842a-445f-a2b1-e6892e9f8aa7 None admin] RBAC: Authorizing `identity:get_user(user_id=07d34fb8652e423dbe23f77fa9277e1b)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:50.279734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b404c-842a-445f-a2b1-e6892e9f8aa7 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:50.287382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 107/216] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:50 2022] GET /identity/v3/users/07d34fb8652e423dbe23f77fa9277e1b => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:50.289407 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b404c-842a-445f-a2b1-e6892e9f8aa7 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:50.300026 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 110/217] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:12:50 2022] POST /identity/v3/users => generated 297 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:12:50.422732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b404c-842a-445f-a2b1-e6892e9f8aa7 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:50.462560 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35933f-89cf-4f3f-a3ca-ab8c83edbc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:50.463182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35933f-89cf-4f3f-a3ca-ab8c83edbc39 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:50.463463 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35933f-89cf-4f3f-a3ca-ab8c83edbc39 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:50.463805 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35933f-89cf-4f3f-a3ca-ab8c83edbc39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:50.493174 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:50.541609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35933f-89cf-4f3f-a3ca-ab8c83edbc39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:50.543014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35933f-89cf-4f3f-a3ca-ab8c83edbc39 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:50.649570 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 108/218] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:12:50.655347 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a54ce9-02d0-4e5d-82e5-e02c58625674 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:50.695761 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:50.781187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a54ce9-02d0-4e5d-82e5-e02c58625674 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:50.818083 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ccc1ca-8766-4c5f-b5d2-2ccfef80b12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:50.818640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ccc1ca-8766-4c5f-b5d2-2ccfef80b12c None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:50.818916 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ccc1ca-8766-4c5f-b5d2-2ccfef80b12c None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:50.819185 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ccc1ca-8766-4c5f-b5d2-2ccfef80b12c None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:50.820379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ccc1ca-8766-4c5f-b5d2-2ccfef80b12c None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:50.821953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ccc1ca-8766-4c5f-b5d2-2ccfef80b12c None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:50.823611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 111/219] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:50 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:50.829754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35933f-89cf-4f3f-a3ca-ab8c83edbc39 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:50.897309 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:50.957172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35933f-89cf-4f3f-a3ca-ab8c83edbc39 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:51.007220 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2cb6d-e259-4ed4-8a58-56e0c9600475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:51.007926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2cb6d-e259-4ed4-8a58-56e0c9600475 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:51.008254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2cb6d-e259-4ed4-8a58-56e0c9600475 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:51.008529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2cb6d-e259-4ed4-8a58-56e0c9600475 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:51.009380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2cb6d-e259-4ed4-8a58-56e0c9600475 None admin] RBAC: Adding query filter params (role.id=246bbd6878334bba893bc6f5bf5caee0, scope.project.id=ad772dbadd1e498f9d58bf65502c4469, user.id=07d34fb8652e423dbe23f77fa9277e1b) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:51.009717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2cb6d-e259-4ed4-8a58-56e0c9600475 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:51.011687 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2cb6d-e259-4ed4-8a58-56e0c9600475 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:51.018287 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 109/220] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:12:50 2022] GET /identity/v3/role_assignments?user.id=07d34fb8652e423dbe23f77fa9277e1b&scope.project.id=ad772dbadd1e498f9d58bf65502c4469&role.id=246bbd6878334bba893bc6f5bf5caee0 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:51.404154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6639dd3-8063-47ef-8125-4782e4dc1145 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:51.404675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6639dd3-8063-47ef-8125-4782e4dc1145 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:51.404675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6639dd3-8063-47ef-8125-4782e4dc1145 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:51.406340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 112/221] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:51.411663 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10607a33-0fa9-40c6-b5ea-9efac454394e None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:51.411953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10607a33-0fa9-40c6-b5ea-9efac454394e None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:51.412257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10607a33-0fa9-40c6-b5ea-9efac454394e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:51.435787 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-10607a33-0fa9-40c6-b5ea-9efac454394e None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:51.437032 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-10607a33-0fa9-40c6-b5ea-9efac454394e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:51.458725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca63c4f-766e-4af1-97b9-ca8b3283d66d None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:51.459091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca63c4f-766e-4af1-97b9-ca8b3283d66d None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:51.459277 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca63c4f-766e-4af1-97b9-ca8b3283d66d None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:51.460890 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 113/222] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:51.466054 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95a9e3-556a-481e-98b5-668cf62f45e9 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:51.466339 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95a9e3-556a-481e-98b5-668cf62f45e9 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:51.466617 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95a9e3-556a-481e-98b5-668cf62f45e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:51.487645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-2e95a9e3-556a-481e-98b5-668cf62f45e9 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:51.488895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-2e95a9e3-556a-481e-98b5-668cf62f45e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:51.547905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 110/223] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:51.571195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10607a33-0fa9-40c6-b5ea-9efac454394e None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:51.616604 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 114/224] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:12:51.628342 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:51.638774 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95a9e3-556a-481e-98b5-668cf62f45e9 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:51.688747 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10607a33-0fa9-40c6-b5ea-9efac454394e None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:51.706674 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:51.736718 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266bc00-5a90-4e3a-b852-6382003bfa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:51.737522 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266bc00-5a90-4e3a-b852-6382003bfa0a None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:51.737850 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266bc00-5a90-4e3a-b852-6382003bfa0a None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:51.738187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266bc00-5a90-4e3a-b852-6382003bfa0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:51.762863 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266bc00-5a90-4e3a-b852-6382003bfa0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:51.770797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-7266bc00-5a90-4e3a-b852-6382003bfa0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:51.770797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:51.770797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:51.770797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:51.770797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:51.770797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:51.770797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:51.770797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:51.770797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:51.770797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:51.770797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:51.770797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:51.770797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:51.770797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:51.770797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:51.770797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:51.771684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:51.771684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:51.771684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:51.771684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:51.771684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:51.771684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:51.771684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:51.771684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:51.771684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:51.771684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:51.771684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:51.771684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:51.771684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:51.771684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:51.771684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:51.771684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:51.772490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:51.772490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:51.772490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:51.772490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:51.772490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:51.772490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:51.772490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:51.772490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:51.772490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:51.772490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:51.772490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:51.772490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:51.775304 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 111/225] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:12:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:51.781607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266bc00-5a90-4e3a-b852-6382003bfa0a None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:51.854392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95a9e3-556a-481e-98b5-668cf62f45e9 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:51.920504 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f01734-40c5-4ebd-b00b-93e79b7e7bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:51.921195 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f01734-40c5-4ebd-b00b-93e79b7e7bef None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:51.921516 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f01734-40c5-4ebd-b00b-93e79b7e7bef None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:51.921833 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f01734-40c5-4ebd-b00b-93e79b7e7bef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:51.934302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f01734-40c5-4ebd-b00b-93e79b7e7bef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:51.939731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-11f01734-40c5-4ebd-b00b-93e79b7e7bef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:12:51.939731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:51.939731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:51.939731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:51.939731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:51.939731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:51.939731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:51.939731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:51.939731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:51.939731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:51.939731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:51.939731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:51.939731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:51.939731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:51.939731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:51.939731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:51.940628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:51.940628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:51.940628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:51.940628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:51.940628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:51.940628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:51.940628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:51.940628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:51.940628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:51.940628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:51.940628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:51.940628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:51.940628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:51.940628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:51.940628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:51.940628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:51.941406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:51.941406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:51.941406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:51.941406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:51.941406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:51.941406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:51.941406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:51.941406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:51.941406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:51.941406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:51.941406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:12:51.941406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:12:51.941406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 115/226] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:12:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:12:51.950932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f01734-40c5-4ebd-b00b-93e79b7e7bef None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:51.967694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266bc00-5a90-4e3a-b852-6382003bfa0a None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:51.979741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:52.006284 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712fe88-5dc3-4510-ab69-13352484f00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:52.006960 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712fe88-5dc3-4510-ab69-13352484f00a None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:52.007242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712fe88-5dc3-4510-ab69-13352484f00a None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:52.007554 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712fe88-5dc3-4510-ab69-13352484f00a None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:52.008258 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712fe88-5dc3-4510-ab69-13352484f00a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:52.008565 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712fe88-5dc3-4510-ab69-13352484f00a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:52.010023 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712fe88-5dc3-4510-ab69-13352484f00a None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:52.019805 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 112/227] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:52.029414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712fe88-5dc3-4510-ab69-13352484f00a None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:52.076511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:52.086456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f01734-40c5-4ebd-b00b-93e79b7e7bef None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:52.124724 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d5c86-2d58-4f7b-849e-dbd8898364e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:52.125367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d5c86-2d58-4f7b-849e-dbd8898364e5 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:52.125644 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d5c86-2d58-4f7b-849e-dbd8898364e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:52.125915 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d5c86-2d58-4f7b-849e-dbd8898364e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:52.127389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d5c86-2d58-4f7b-849e-dbd8898364e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:52.127389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d5c86-2d58-4f7b-849e-dbd8898364e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:52.128379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8d5c86-2d58-4f7b-849e-dbd8898364e5 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:52.137893 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 116/228] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:12:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:52.149085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d5c86-2d58-4f7b-849e-dbd8898364e5 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:52.165195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712fe88-5dc3-4510-ab69-13352484f00a None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:52.202918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191842d-a696-4bd1-8c0b-c26f103e014c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:52.203935 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191842d-a696-4bd1-8c0b-c26f103e014c None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:52.204441 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191842d-a696-4bd1-8c0b-c26f103e014c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:52.204907 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191842d-a696-4bd1-8c0b-c26f103e014c None admin] PATH_INFO: `/v3/users/67cbfba3427c42ba9f403be29873efd9` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:52.206921 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191842d-a696-4bd1-8c0b-c26f103e014c None admin] RBAC: Authorizing `identity:get_user(user_id=67cbfba3427c42ba9f403be29873efd9)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:52.208668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191842d-a696-4bd1-8c0b-c26f103e014c None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:52.210750 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 113/229] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:52 2022] GET /identity/v3/users/67cbfba3427c42ba9f403be29873efd9 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:52.218867 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191842d-a696-4bd1-8c0b-c26f103e014c None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:52.264533 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:52.333566 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191842d-a696-4bd1-8c0b-c26f103e014c None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:52.371938 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0804ca7-2e1d-4c5b-8b9e-db2dbd23315f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:52.372674 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0804ca7-2e1d-4c5b-8b9e-db2dbd23315f None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:52.373374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0804ca7-2e1d-4c5b-8b9e-db2dbd23315f None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:52.373650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0804ca7-2e1d-4c5b-8b9e-db2dbd23315f None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:52.374924 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0804ca7-2e1d-4c5b-8b9e-db2dbd23315f None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:52.376564 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0804ca7-2e1d-4c5b-8b9e-db2dbd23315f None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:52.378196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d5c86-2d58-4f7b-849e-dbd8898364e5 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:52.378647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 114/230] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:52 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:52.386016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0804ca7-2e1d-4c5b-8b9e-db2dbd23315f None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:52.473367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75a567-9442-4b13-bee8-76d45edc569d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:52.474013 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75a567-9442-4b13-bee8-76d45edc569d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:52.474200 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75a567-9442-4b13-bee8-76d45edc569d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:52.474466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac75a567-9442-4b13-bee8-76d45edc569d None admin] PATH_INFO: `/v3/users/67cbfba3427c42ba9f403be29873efd9` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:52.478565 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75a567-9442-4b13-bee8-76d45edc569d None admin] RBAC: Authorizing `identity:get_user(user_id=67cbfba3427c42ba9f403be29873efd9)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:52.480082 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac75a567-9442-4b13-bee8-76d45edc569d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:52.481756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 117/231] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:52 2022] GET /identity/v3/users/67cbfba3427c42ba9f403be29873efd9 => generated 283 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:52.487331 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75a567-9442-4b13-bee8-76d45edc569d None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:52.502010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0804ca7-2e1d-4c5b-8b9e-db2dbd23315f None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:52.512860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:52.536596 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:52.542711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bc7f1-4a14-4f41-8d5b-3eca05a8e149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:52.543423 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bc7f1-4a14-4f41-8d5b-3eca05a8e149 None admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:52.543898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bc7f1-4a14-4f41-8d5b-3eca05a8e149 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:52.544182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bc7f1-4a14-4f41-8d5b-3eca05a8e149 None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469/users/67cbfba3427c42ba9f403be29873efd9/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:52.546152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391bc7f1-4a14-4f41-8d5b-3eca05a8e149 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad772dbadd1e498f9d58bf65502c4469, user_id=67cbfba3427c42ba9f403be29873efd9, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:52.547603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391bc7f1-4a14-4f41-8d5b-3eca05a8e149 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:52.566650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 115/232] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:12:52 2022] PUT /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469/users/67cbfba3427c42ba9f403be29873efd9/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:12:52.679186 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac75a567-9442-4b13-bee8-76d45edc569d None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:52.727467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297be82-ed54-4903-8c3e-3f3b045ee9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:52.728099 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297be82-ed54-4903-8c3e-3f3b045ee9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:52.728382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297be82-ed54-4903-8c3e-3f3b045ee9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:52.728654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297be82-ed54-4903-8c3e-3f3b045ee9c7 None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:52.729892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2297be82-ed54-4903-8c3e-3f3b045ee9c7 None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:52.731579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2297be82-ed54-4903-8c3e-3f3b045ee9c7 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:52.733759 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 118/233] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:52 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:52.741202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bc7f1-4a14-4f41-8d5b-3eca05a8e149 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:52.863595 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bc7f1-4a14-4f41-8d5b-3eca05a8e149 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:52.904407 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead87373-bd49-427b-aff2-96818d0b33cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:52.905541 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead87373-bd49-427b-aff2-96818d0b33cd None admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:52.906018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead87373-bd49-427b-aff2-96818d0b33cd None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:52.906498 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead87373-bd49-427b-aff2-96818d0b33cd None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469/users/67cbfba3427c42ba9f403be29873efd9/roles/9a1c954216e34607a6788f44f07a593b` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:52.910351 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead87373-bd49-427b-aff2-96818d0b33cd None admin] RBAC: Authorizing `identity:create_grant(project_id=ad772dbadd1e498f9d58bf65502c4469, user_id=67cbfba3427c42ba9f403be29873efd9, role_id=9a1c954216e34607a6788f44f07a593b)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:52.912822 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead87373-bd49-427b-aff2-96818d0b33cd None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:52.935912 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 116/234] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:12:52 2022] PUT /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469/users/67cbfba3427c42ba9f403be29873efd9/roles/9a1c954216e34607a6788f44f07a593b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:12:53.150696 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b58f84-0471-4a5e-985c-0a5f26ab2236 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:53.151070 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b58f84-0471-4a5e-985c-0a5f26ab2236 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:53.151317 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b58f84-0471-4a5e-985c-0a5f26ab2236 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:53.153244 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 119/235] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:53.158863 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acab2de-8f68-4c01-baf1-6ea4a6b89458 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:53.159395 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acab2de-8f68-4c01-baf1-6ea4a6b89458 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:53.159395 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acab2de-8f68-4c01-baf1-6ea4a6b89458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:53.182420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-9acab2de-8f68-4c01-baf1-6ea4a6b89458 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:53.184219 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9acab2de-8f68-4c01-baf1-6ea4a6b89458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:53.215267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:53.274873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 117/236] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:53.302583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b58f84-0471-4a5e-985c-0a5f26ab2236 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:53.329178 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dd8de-b4ff-45cb-bab9-e06f54b62783 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:53.329527 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dd8de-b4ff-45cb-bab9-e06f54b62783 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:53.329724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523dd8de-b4ff-45cb-bab9-e06f54b62783 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:53.331468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 118/237] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:12:53.336870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78d3da-4567-49c9-a229-874324a11880 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:53.337156 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78d3da-4567-49c9-a229-874324a11880 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:53.337433 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78d3da-4567-49c9-a229-874324a11880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:53.356637 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-1f78d3da-4567-49c9-a229-874324a11880 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:53.357854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1f78d3da-4567-49c9-a229-874324a11880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:53.466742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 119/238] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:12:53.491818 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d9267-02c2-42ff-98a1-93702660e3ca None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:53.492169 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d9267-02c2-42ff-98a1-93702660e3ca None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:53.492425 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d9267-02c2-42ff-98a1-93702660e3ca None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:53.493917 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 120/239] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:53.498256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d9267-02c2-42ff-98a1-93702660e3ca None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:53.512406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:53.516527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b58f84-0471-4a5e-985c-0a5f26ab2236 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:53.551294 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f313a-211d-490b-a052-0df2780a637c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:53.552294 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f313a-211d-490b-a052-0df2780a637c None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:53.552761 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f313a-211d-490b-a052-0df2780a637c None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:53.553213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f313a-211d-490b-a052-0df2780a637c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:53.565230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f313a-211d-490b-a052-0df2780a637c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:53.570328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-2e8f313a-211d-490b-a052-0df2780a637c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:12:53.570328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:53.570328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:53.570328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:53.570328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:53.570328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:53.570328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:53.570328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:53.570328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:53.570328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:53.570328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:53.570328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:53.570328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:53.570328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:53.570328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:53.570328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:53.571226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:53.571226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:53.571226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:53.571226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:53.571226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:53.571226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:53.571226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:53.571226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:53.571226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:53.571226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:53.571226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:53.571226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:53.571226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:53.571226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:53.571226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:53.571226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:53.572040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:53.572040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:53.572040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:53.572040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:53.572040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:53.572040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:53.572040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:53.572040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:53.572040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:53.572040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:53.572040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:12:53.572040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:12:53.574768 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 120/240] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:53.576295 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc2fae-9ef9-4478-b449-1492f6dca542 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:53.576829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc2fae-9ef9-4478-b449-1492f6dca542 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:53.577286 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc2fae-9ef9-4478-b449-1492f6dca542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:53.595752 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-f2dc2fae-9ef9-4478-b449-1492f6dca542 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:53.597352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f2dc2fae-9ef9-4478-b449-1492f6dca542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:53.605364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d9267-02c2-42ff-98a1-93702660e3ca None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:53.608396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:53.642581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703e684-8085-4ad4-94ef-28cca90f5713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:53.643232 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703e684-8085-4ad4-94ef-28cca90f5713 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:53.643511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703e684-8085-4ad4-94ef-28cca90f5713 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:53.643867 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703e684-8085-4ad4-94ef-28cca90f5713 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:53.645019 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703e684-8085-4ad4-94ef-28cca90f5713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:53.646480 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703e684-8085-4ad4-94ef-28cca90f5713 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:53.648207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 121/241] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:12:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:53.648977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703e684-8085-4ad4-94ef-28cca90f5713 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:53.671514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 121/242] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:53.672924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc2fae-9ef9-4478-b449-1492f6dca542 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:53.759873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703e684-8085-4ad4-94ef-28cca90f5713 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:53.768425 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:53.790964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc2fae-9ef9-4478-b449-1492f6dca542 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:53.792860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd7840-f450-4268-b32a-366ac62cfba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:53.793558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd7840-f450-4268-b32a-366ac62cfba1 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:53.793834 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd7840-f450-4268-b32a-366ac62cfba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:53.794104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd7840-f450-4268-b32a-366ac62cfba1 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:53.795170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd7840-f450-4268-b32a-366ac62cfba1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:53.795170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd7840-f450-4268-b32a-366ac62cfba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:53.796464 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd7840-f450-4268-b32a-366ac62cfba1 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:53.805278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 122/243] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:12:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:53.811044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd7840-f450-4268-b32a-366ac62cfba1 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:53.831092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:53.831477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2828473-0e23-46bf-9e3e-7929de283d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:53.832468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2828473-0e23-46bf-9e3e-7929de283d34 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:53.832965 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2828473-0e23-46bf-9e3e-7929de283d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:53.833419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2828473-0e23-46bf-9e3e-7929de283d34 None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:53.835103 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2828473-0e23-46bf-9e3e-7929de283d34 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:53.836943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2828473-0e23-46bf-9e3e-7929de283d34 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:53.861601 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 122/244] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:12:53 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:12:53.862944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2828473-0e23-46bf-9e3e-7929de283d34 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:53.918371 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd7840-f450-4268-b32a-366ac62cfba1 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:53.954307 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec654b84-1aad-4d43-8c23-6983c5477e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:53.955194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec654b84-1aad-4d43-8c23-6983c5477e50 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:53.955609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec654b84-1aad-4d43-8c23-6983c5477e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:53.956090 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec654b84-1aad-4d43-8c23-6983c5477e50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:53.969493 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec654b84-1aad-4d43-8c23-6983c5477e50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:53.975009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-ec654b84-1aad-4d43-8c23-6983c5477e50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:53.975009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:53.975009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:53.975009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:53.975009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:53.975009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:53.975009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:53.975009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:53.975009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:53.975009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:53.975009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:53.975009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:53.975009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:53.975009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:53.975009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:53.975009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:53.976010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:53.976010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:53.976010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:53.976010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:53.976010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:53.976010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:53.976010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:53.976010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:53.976010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:53.976010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:53.976010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:53.976010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:53.976010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:53.976010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:53.976010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:53.976010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:53.976806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:53.976806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:53.976806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:53.976806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:53.976806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:53.976806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:53.976806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:53.976806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:53.976806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:53.976806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:53.976806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:53.976806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:53.979269 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 123/245] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:53.988974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2828473-0e23-46bf-9e3e-7929de283d34 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:53.999712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec654b84-1aad-4d43-8c23-6983c5477e50 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:54.041255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33d6be-b66b-4d75-bd5f-283138f02861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:54.041920 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33d6be-b66b-4d75-bd5f-283138f02861 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:54.042228 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33d6be-b66b-4d75-bd5f-283138f02861 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:54.042458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33d6be-b66b-4d75-bd5f-283138f02861 None admin] PATH_INFO: `/v3/users/aed21a5dcdc949b9a4e96ba85f023216` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:54.045392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:54.083159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33d6be-b66b-4d75-bd5f-283138f02861 None admin] RBAC: Authorizing `identity:get_user(user_id=aed21a5dcdc949b9a4e96ba85f023216)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:54.085033 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33d6be-b66b-4d75-bd5f-283138f02861 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:54.091125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 123/246] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:53 2022] GET /identity/v3/users/aed21a5dcdc949b9a4e96ba85f023216 => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:54.101212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33d6be-b66b-4d75-bd5f-283138f02861 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:54.119368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:54.148331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec654b84-1aad-4d43-8c23-6983c5477e50 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:54.189610 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef5131-afee-4914-b0ae-cd24dac40d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:54.190230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef5131-afee-4914-b0ae-cd24dac40d79 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:54.190500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef5131-afee-4914-b0ae-cd24dac40d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:54.190774 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef5131-afee-4914-b0ae-cd24dac40d79 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:54.191562 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef5131-afee-4914-b0ae-cd24dac40d79 None admin] RBAC: Adding query filter params (name=member) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:54.191867 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef5131-afee-4914-b0ae-cd24dac40d79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:54.193321 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef5131-afee-4914-b0ae-cd24dac40d79 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:54.203236 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 124/247] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:12:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:54.209295 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef5131-afee-4914-b0ae-cd24dac40d79 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:54.235794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33d6be-b66b-4d75-bd5f-283138f02861 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:54.283704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43098d05-54c4-41f7-8297-77b32b525f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:54.285304 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43098d05-54c4-41f7-8297-77b32b525f2d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:54.285921 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43098d05-54c4-41f7-8297-77b32b525f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:54.286459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43098d05-54c4-41f7-8297-77b32b525f2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:54.335614 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef5131-afee-4914-b0ae-cd24dac40d79 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:54.363230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:54.376458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43098d05-54c4-41f7-8297-77b32b525f2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:54.377333 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58be23-86bd-4d57-855a-780f6adda652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:54.378475 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43098d05-54c4-41f7-8297-77b32b525f2d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:54.380353 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58be23-86bd-4d57-855a-780f6adda652 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:54.380761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58be23-86bd-4d57-855a-780f6adda652 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:54.381120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e58be23-86bd-4d57-855a-780f6adda652 None admin] PATH_INFO: `/v3/users/07d34fb8652e423dbe23f77fa9277e1b` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:54.382438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e58be23-86bd-4d57-855a-780f6adda652 None admin] RBAC: Authorizing `identity:get_user(user_id=07d34fb8652e423dbe23f77fa9277e1b)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:54.384025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e58be23-86bd-4d57-855a-780f6adda652 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:54.386144 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 125/248] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:54 2022] GET /identity/v3/users/07d34fb8652e423dbe23f77fa9277e1b => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:54.394580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58be23-86bd-4d57-855a-780f6adda652 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:54.411037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:54.514992 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 124/249] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 19:12:54.520726 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43098d05-54c4-41f7-8297-77b32b525f2d None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:54.524364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e58be23-86bd-4d57-855a-780f6adda652 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:54.561216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d1883-8302-4792-8a42-7843540e8890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:54.562163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d1883-8302-4792-8a42-7843540e8890 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:54.562613 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d1883-8302-4792-8a42-7843540e8890 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:54.563080 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d1883-8302-4792-8a42-7843540e8890 None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:54.564424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d1883-8302-4792-8a42-7843540e8890 None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:54.566056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d1883-8302-4792-8a42-7843540e8890 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:54.568226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 126/250] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:54 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:54.581950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d1883-8302-4792-8a42-7843540e8890 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:54.626144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:54.642677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43098d05-54c4-41f7-8297-77b32b525f2d None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:54.678738 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:54.681570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df8c95-7299-49d7-81e6-078828a50ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:54.682452 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df8c95-7299-49d7-81e6-078828a50ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:54.683295 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df8c95-7299-49d7-81e6-078828a50ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:54.683767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df8c95-7299-49d7-81e6-078828a50ca1 None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:54.685913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df8c95-7299-49d7-81e6-078828a50ca1 None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:54.687579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df8c95-7299-49d7-81e6-078828a50ca1 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:54.690065 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 125/251] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:54 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:54.698078 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df8c95-7299-49d7-81e6-078828a50ca1 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:54.704732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d1883-8302-4792-8a42-7843540e8890 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:54.741178 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcaf9d-ae77-44ac-a124-1d3621eecffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:54.741864 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcaf9d-ae77-44ac-a124-1d3621eecffe None admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:54.742146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcaf9d-ae77-44ac-a124-1d3621eecffe None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:54.742420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcaf9d-ae77-44ac-a124-1d3621eecffe None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469/users/07d34fb8652e423dbe23f77fa9277e1b/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:54.744547 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcaf9d-ae77-44ac-a124-1d3621eecffe None admin] RBAC: Authorizing `identity:create_grant(project_id=ad772dbadd1e498f9d58bf65502c4469, user_id=07d34fb8652e423dbe23f77fa9277e1b, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:54.746185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcaf9d-ae77-44ac-a124-1d3621eecffe None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:54.761403 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 127/252] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:12:54 2022] PUT /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469/users/07d34fb8652e423dbe23f77fa9277e1b/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:12:54.826064 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df8c95-7299-49d7-81e6-078828a50ca1 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:54.866684 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e29fd-d934-4ae3-ac19-8e6c3839a97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:54.867407 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e29fd-d934-4ae3-ac19-8e6c3839a97d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:54.867782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e29fd-d934-4ae3-ac19-8e6c3839a97d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:54.868146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e29fd-d934-4ae3-ac19-8e6c3839a97d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:54.868970 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e29fd-d934-4ae3-ac19-8e6c3839a97d None admin] RBAC: Adding query filter params (role.id=31d708bdce024bbc8423f085d27b1776, scope.project.id=ad772dbadd1e498f9d58bf65502c4469, user.id=aed21a5dcdc949b9a4e96ba85f023216) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:54.869276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e29fd-d934-4ae3-ac19-8e6c3839a97d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:54.870913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e29fd-d934-4ae3-ac19-8e6c3839a97d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:54.878346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 126/253] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:12:54 2022] GET /identity/v3/role_assignments?user.id=aed21a5dcdc949b9a4e96ba85f023216&scope.project.id=ad772dbadd1e498f9d58bf65502c4469&role.id=31d708bdce024bbc8423f085d27b1776 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:55.283842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cc164-ae44-47c5-bb66-a3ae5c29b03a None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:55.284412 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cc164-ae44-47c5-bb66-a3ae5c29b03a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:55.284412 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9cc164-ae44-47c5-bb66-a3ae5c29b03a None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:55.285935 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 128/254] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:55.291415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88c412-457e-4471-9cbe-f57bb1ea687f None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:55.291750 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88c412-457e-4471-9cbe-f57bb1ea687f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:55.292046 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88c412-457e-4471-9cbe-f57bb1ea687f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:55.311193 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-2f88c412-457e-4471-9cbe-f57bb1ea687f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:55.316578 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-2f88c412-457e-4471-9cbe-f57bb1ea687f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:55.340315 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:55.410935 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 127/255] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:55.446920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9cc164-ae44-47c5-bb66-a3ae5c29b03a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:55.566469 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9cc164-ae44-47c5-bb66-a3ae5c29b03a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:55.606611 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4edc7-2e34-47c0-a129-9bcad678b920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:55.607277 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4edc7-2e34-47c0-a129-9bcad678b920 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:55.607557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4edc7-2e34-47c0-a129-9bcad678b920 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:55.607826 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4edc7-2e34-47c0-a129-9bcad678b920 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:55.613854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:55.619066 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bd8a0-2e76-4f84-8006-ad35554c963f None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:55.619591 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bd8a0-2e76-4f84-8006-ad35554c963f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:55.619663 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bd8a0-2e76-4f84-8006-ad35554c963f None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:55.621187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 128/256] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:12:55.627494 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939711b-788b-47eb-9831-16e2a586e044 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:55.627813 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939711b-788b-47eb-9831-16e2a586e044 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:55.628122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939711b-788b-47eb-9831-16e2a586e044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:55.640601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4edc7-2e34-47c0-a129-9bcad678b920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:55.646373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-2ee4edc7-2e34-47c0-a129-9bcad678b920 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:55.646373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:55.646373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:55.646373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:55.646373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:55.646373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:55.646373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:55.646373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:55.646373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:55.646373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:55.646373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:55.646373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:55.646373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:55.646373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:55.646373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:55.646373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:55.647343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:55.647343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:55.647343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:55.647343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:55.647343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:55.647343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:55.647343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:55.647343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:55.647343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:55.647343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:55.647343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:55.647343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:55.647343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:55.647343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:55.647343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:55.647343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:55.648194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:55.648194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:55.648194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:55.648194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:55.648194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:55.648194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:55.648194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:55.648194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:55.648194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:55.648194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:55.648194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:12:55.648194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:55.648798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 129/257] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:12:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:55.656037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4edc7-2e34-47c0-a129-9bcad678b920 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:55.661071 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-9939711b-788b-47eb-9831-16e2a586e044 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:55.662298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9939711b-788b-47eb-9831-16e2a586e044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:55.778094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4edc7-2e34-47c0-a129-9bcad678b920 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:55.818148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efa9c8-181c-402a-bb64-d0cc50aef2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:55.818681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efa9c8-181c-402a-bb64-d0cc50aef2ed None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:55.818962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efa9c8-181c-402a-bb64-d0cc50aef2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:55.819235 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efa9c8-181c-402a-bb64-d0cc50aef2ed None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:55.819967 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efa9c8-181c-402a-bb64-d0cc50aef2ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:55.820299 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efa9c8-181c-402a-bb64-d0cc50aef2ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:55.822005 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efa9c8-181c-402a-bb64-d0cc50aef2ed None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:55.832062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 130/258] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:55.837823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efa9c8-181c-402a-bb64-d0cc50aef2ed None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:55.851718 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 129/259] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:55.879407 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939711b-788b-47eb-9831-16e2a586e044 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:55.983586 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:56.001736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:56.005169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939711b-788b-47eb-9831-16e2a586e044 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:56.046299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f250c86-02ac-41dd-92a8-5be12764e96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:56.047234 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f250c86-02ac-41dd-92a8-5be12764e96c None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:56.047754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f250c86-02ac-41dd-92a8-5be12764e96c None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:56.048493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f250c86-02ac-41dd-92a8-5be12764e96c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:56.060258 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f250c86-02ac-41dd-92a8-5be12764e96c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:56.065510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-8f250c86-02ac-41dd-92a8-5be12764e96c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:12:56.065510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:56.065510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:56.065510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:56.065510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:56.065510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:56.065510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:56.065510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:56.065510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:56.065510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:56.065510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:56.065510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:56.065510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:56.065510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:56.065510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:56.065510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:56.066460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:56.066460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:56.066460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:56.066460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:56.066460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:56.066460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:56.066460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:56.066460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:56.066460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:56.066460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:56.066460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:56.066460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:56.066460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:56.066460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:56.066460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:56.066460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:56.067332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:56.067332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:56.067332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:56.067332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:56.067332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:56.067332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:56.067332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:56.067332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:56.067332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:56.067332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:56.067332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:12:56.067332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:12:56.070198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 130/260] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:12:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:12:56.076226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f250c86-02ac-41dd-92a8-5be12764e96c None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:56.094156 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efa9c8-181c-402a-bb64-d0cc50aef2ed None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:56.179175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1535d78-5686-4d57-a46d-642375eec626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:56.179926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1535d78-5686-4d57-a46d-642375eec626 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:56.180311 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1535d78-5686-4d57-a46d-642375eec626 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:56.180636 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1535d78-5686-4d57-a46d-642375eec626 None admin] PATH_INFO: `/v3/users/67cbfba3427c42ba9f403be29873efd9` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:56.181941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1535d78-5686-4d57-a46d-642375eec626 None admin] RBAC: Authorizing `identity:get_user(user_id=67cbfba3427c42ba9f403be29873efd9)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:56.183794 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1535d78-5686-4d57-a46d-642375eec626 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:56.184892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f250c86-02ac-41dd-92a8-5be12764e96c None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:56.192475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 131/261] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:55 2022] GET /identity/v3/users/67cbfba3427c42ba9f403be29873efd9 => generated 283 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:56.193494 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1535d78-5686-4d57-a46d-642375eec626 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:56.230864 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e0336-f664-4820-ba9b-798c74b76284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:56.231852 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e0336-f664-4820-ba9b-798c74b76284 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:56.232318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e0336-f664-4820-ba9b-798c74b76284 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:56.232759 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e0336-f664-4820-ba9b-798c74b76284 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:56.233732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e0336-f664-4820-ba9b-798c74b76284 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:56.234198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e0336-f664-4820-ba9b-798c74b76284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:56.235823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e0336-f664-4820-ba9b-798c74b76284 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:56.252254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 131/262] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:12:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:56.258553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e0336-f664-4820-ba9b-798c74b76284 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:56.276898 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:56.381476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e0336-f664-4820-ba9b-798c74b76284 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:56.423023 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1535d78-5686-4d57-a46d-642375eec626 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:56.427947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06638de7-3489-432a-b61c-09664e197753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:56.428882 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06638de7-3489-432a-b61c-09664e197753 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:56.429332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06638de7-3489-432a-b61c-09664e197753 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:56.429772 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06638de7-3489-432a-b61c-09664e197753 None admin] PATH_INFO: `/v3/users/67cbfba3427c42ba9f403be29873efd9` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:56.431090 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06638de7-3489-432a-b61c-09664e197753 None admin] RBAC: Authorizing `identity:get_user(user_id=67cbfba3427c42ba9f403be29873efd9)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:56.433441 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06638de7-3489-432a-b61c-09664e197753 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:56.435397 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 132/263] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:56 2022] GET /identity/v3/users/67cbfba3427c42ba9f403be29873efd9 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:56.441349 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06638de7-3489-432a-b61c-09664e197753 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:56.469046 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:56.484311 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c66c56-434a-43bb-8853-24ce2e0ae252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:56.484942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66c56-434a-43bb-8853-24ce2e0ae252 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:56.488382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66c56-434a-43bb-8853-24ce2e0ae252 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:56.488663 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c66c56-434a-43bb-8853-24ce2e0ae252 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:56.527871 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:56.556541 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06638de7-3489-432a-b61c-09664e197753 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:56.589791 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c66c56-434a-43bb-8853-24ce2e0ae252 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:56.591415 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c66c56-434a-43bb-8853-24ce2e0ae252 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:56.594710 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aaaab5-9f14-4381-904a-37df55308d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:56.595614 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aaaab5-9f14-4381-904a-37df55308d03 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:56.596144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aaaab5-9f14-4381-904a-37df55308d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:56.596636 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aaaab5-9f14-4381-904a-37df55308d03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:56.671587 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aaaab5-9f14-4381-904a-37df55308d03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:56.674043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aaaab5-9f14-4381-904a-37df55308d03 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:56.691692 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 132/264] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:12:56.697466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c66c56-434a-43bb-8853-24ce2e0ae252 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:56.725888 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:56.775348 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:56.799677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c66c56-434a-43bb-8853-24ce2e0ae252 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:56.806856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 133/265] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 19:12:56.813219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aaaab5-9f14-4381-904a-37df55308d03 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:56.834859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6a2dc-3faa-49eb-955c-cb45b1c04829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:56.835490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6a2dc-3faa-49eb-955c-cb45b1c04829 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:56.835764 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6a2dc-3faa-49eb-955c-cb45b1c04829 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:56.836056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6a2dc-3faa-49eb-955c-cb45b1c04829 None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:56.837467 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6a2dc-3faa-49eb-955c-cb45b1c04829 None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:56.838657 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6a2dc-3faa-49eb-955c-cb45b1c04829 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:56.843733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 133/266] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:56 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:56.846512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6a2dc-3faa-49eb-955c-cb45b1c04829 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:56.923037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aaaab5-9f14-4381-904a-37df55308d03 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:56.939000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:56.949545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6a2dc-3faa-49eb-955c-cb45b1c04829 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:56.965018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a83ffc-ed0d-4497-b688-da0215fc7e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:56.966153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a83ffc-ed0d-4497-b688-da0215fc7e43 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:56.966672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a83ffc-ed0d-4497-b688-da0215fc7e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:56.967245 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a83ffc-ed0d-4497-b688-da0215fc7e43 None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:56.968646 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a83ffc-ed0d-4497-b688-da0215fc7e43 None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:56.970393 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a83ffc-ed0d-4497-b688-da0215fc7e43 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:56.972606 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 134/267] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:56 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:56.979307 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a83ffc-ed0d-4497-b688-da0215fc7e43 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:56.985600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480d27f-d07b-4858-8942-e7364e4f8f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:56.986217 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480d27f-d07b-4858-8942-e7364e4f8f94 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:56.986490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480d27f-d07b-4858-8942-e7364e4f8f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:56.986757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480d27f-d07b-4858-8942-e7364e4f8f94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:56.987492 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480d27f-d07b-4858-8942-e7364e4f8f94 None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=ad772dbadd1e498f9d58bf65502c4469, user.id=67cbfba3427c42ba9f403be29873efd9) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:56.987797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480d27f-d07b-4858-8942-e7364e4f8f94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:56.989311 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480d27f-d07b-4858-8942-e7364e4f8f94 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:57.000120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 134/268] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:12:56 2022] GET /identity/v3/role_assignments?user.id=67cbfba3427c42ba9f403be29873efd9&scope.project.id=ad772dbadd1e498f9d58bf65502c4469&role.id=4da42f08d107448fb877b42b61bc74ea => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:57.032232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:57.105635 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a83ffc-ed0d-4497-b688-da0215fc7e43 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:57.145773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c6a82-7b9f-4be3-b7b5-9c384ffc6fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:57.146474 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c6a82-7b9f-4be3-b7b5-9c384ffc6fca None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:57.146747 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c6a82-7b9f-4be3-b7b5-9c384ffc6fca None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:57.147017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c6a82-7b9f-4be3-b7b5-9c384ffc6fca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:57.147773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c6a82-7b9f-4be3-b7b5-9c384ffc6fca None admin] RBAC: Adding query filter params (role.id=9a1c954216e34607a6788f44f07a593b, scope.project.id=ad772dbadd1e498f9d58bf65502c4469, user.id=67cbfba3427c42ba9f403be29873efd9) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:57.148178 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c6a82-7b9f-4be3-b7b5-9c384ffc6fca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:57.149713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c6a82-7b9f-4be3-b7b5-9c384ffc6fca None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:57.156984 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 135/269] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:12:56 2022] GET /identity/v3/role_assignments?user.id=67cbfba3427c42ba9f403be29873efd9&scope.project.id=ad772dbadd1e498f9d58bf65502c4469&role.id=9a1c954216e34607a6788f44f07a593b => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:57.688561 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb616b8-b666-4690-9e71-302955355369 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:57.689561 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb616b8-b666-4690-9e71-302955355369 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:57.690051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb616b8-b666-4690-9e71-302955355369 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:57.691588 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 135/270] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:57.697149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494b94e-af7d-46ec-b95e-c10f9283a979 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:57.697981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494b94e-af7d-46ec-b95e-c10f9283a979 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:57.698443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494b94e-af7d-46ec-b95e-c10f9283a979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:57.720390 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-8494b94e-af7d-46ec-b95e-c10f9283a979 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:57.721913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8494b94e-af7d-46ec-b95e-c10f9283a979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:57.805514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 136/271] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:12:57.816814 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062644cc-6a0d-4a6d-b4d1-aab5f69a8c15 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:57.817470 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062644cc-6a0d-4a6d-b4d1-aab5f69a8c15 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:57.818099 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062644cc-6a0d-4a6d-b4d1-aab5f69a8c15 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:57.820124 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 136/272] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:57.826419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38651ba-5884-49f1-b6b3-e418a3b0e841 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:57.826777 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38651ba-5884-49f1-b6b3-e418a3b0e841 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:57.827112 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38651ba-5884-49f1-b6b3-e418a3b0e841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:57.837376 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062644cc-6a0d-4a6d-b4d1-aab5f69a8c15 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:57.849795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-c38651ba-5884-49f1-b6b3-e418a3b0e841 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:57.851524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c38651ba-5884-49f1-b6b3-e418a3b0e841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:57.891763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:57.937137 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 137/273] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:57.938236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db2d8e-e372-4c27-bd8b-809732658773 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:57.938516 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db2d8e-e372-4c27-bd8b-809732658773 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:57.938844 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db2d8e-e372-4c27-bd8b-809732658773 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:57.940239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 138/274] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:57.949543 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f53438-900d-40ee-ac5c-a72c52cfb7de None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:57.949895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f53438-900d-40ee-ac5c-a72c52cfb7de None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:57.950120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f53438-900d-40ee-ac5c-a72c52cfb7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:57.953015 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062644cc-6a0d-4a6d-b4d1-aab5f69a8c15 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:57.969379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-59f53438-900d-40ee-ac5c-a72c52cfb7de None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:57.970615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-59f53438-900d-40ee-ac5c-a72c52cfb7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:57.993947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85877f0d-30fc-4298-97c8-c58707de7880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:57.994869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85877f0d-30fc-4298-97c8-c58707de7880 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:57.995197 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85877f0d-30fc-4298-97c8-c58707de7880 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:57.995551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85877f0d-30fc-4298-97c8-c58707de7880 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:57.997115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85877f0d-30fc-4298-97c8-c58707de7880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:57.998715 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85877f0d-30fc-4298-97c8-c58707de7880 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:58.000994 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 137/275] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:12:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:12:58.002225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85877f0d-30fc-4298-97c8-c58707de7880 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:58.034378 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:58.063699 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 139/276] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:58.064674 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f53438-900d-40ee-ac5c-a72c52cfb7de None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:58.126919 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85877f0d-30fc-4298-97c8-c58707de7880 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:58.161175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0df79-fa9e-48d1-bf27-eb8dc602949d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:58.161822 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0df79-fa9e-48d1-bf27-eb8dc602949d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:58.162105 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0df79-fa9e-48d1-bf27-eb8dc602949d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:58.162381 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0df79-fa9e-48d1-bf27-eb8dc602949d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:58.173071 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0df79-fa9e-48d1-bf27-eb8dc602949d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:58.178202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-36a0df79-fa9e-48d1-bf27-eb8dc602949d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:58.178202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:58.178202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:58.178202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:58.178202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:58.178202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:58.178202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:58.178202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:58.178202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:58.178202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:58.178202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:58.178202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:58.178202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:58.178202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:58.178202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:58.178202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:58.179327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:58.179327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:58.179327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:58.179327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:58.179327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:58.179327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:58.179327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:58.179327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:58.179327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:58.179327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:58.179327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:58.179327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:58.179327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:58.179327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:58.179327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:58.179327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:58.180428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:58.180428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:58.180428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:58.180428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:58.180428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:58.180428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:58.180428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:58.180428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:58.180428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:58.180428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:58.180428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:58.180428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:58.180428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 138/277] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:58.180428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0df79-fa9e-48d1-bf27-eb8dc602949d None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:58.182134 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:58.192437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f53438-900d-40ee-ac5c-a72c52cfb7de None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:58.229452 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922abb7-29b7-42bc-a6f1-c77a166343aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:58.230069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922abb7-29b7-42bc-a6f1-c77a166343aa None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:58.230364 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922abb7-29b7-42bc-a6f1-c77a166343aa None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:58.230617 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922abb7-29b7-42bc-a6f1-c77a166343aa None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:58.231316 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922abb7-29b7-42bc-a6f1-c77a166343aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:58.232815 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922abb7-29b7-42bc-a6f1-c77a166343aa None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:58.253811 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 140/278] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:12:58 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:12:58.255072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e79a8d-b2e7-4ee7-8cc7-11140ff7fa50 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:58.255355 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e79a8d-b2e7-4ee7-8cc7-11140ff7fa50 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:58.255629 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e79a8d-b2e7-4ee7-8cc7-11140ff7fa50 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:58.261429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 141/279] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:12:58.261429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e79a8d-b2e7-4ee7-8cc7-11140ff7fa50 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:58.299567 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:58.317904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0df79-fa9e-48d1-bf27-eb8dc602949d None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:58.359214 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e26d5-1162-4d21-812b-3a270698d4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:58.360026 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e26d5-1162-4d21-812b-3a270698d4dd None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:58.360302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e26d5-1162-4d21-812b-3a270698d4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:58.360580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e26d5-1162-4d21-812b-3a270698d4dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:58.373621 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e26d5-1162-4d21-812b-3a270698d4dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:58.381678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-eb5e26d5-1162-4d21-812b-3a270698d4dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:12:58.381678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:58.381678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:58.381678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:58.381678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:58.381678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:58.381678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:58.381678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:58.381678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:58.381678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:58.381678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:58.381678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:58.381678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:58.381678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:58.381678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:58.381678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:58.382688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:58.382688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:58.382688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:58.382688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:58.382688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:58.382688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:58.382688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:58.382688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:58.382688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:58.382688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:58.382688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:58.382688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:58.382688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:58.382688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:58.382688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:58.382688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:58.383923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:58.383923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:58.383923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:58.383923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:58.383923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:58.383923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:58.383923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:58.383923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:58.383923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:58.383923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:58.383923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:12:58.383923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:12:58.383923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 139/280] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:58.385866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d91a-ca88-49fe-bc72-c6adeb2abb24 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:58.386003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d91a-ca88-49fe-bc72-c6adeb2abb24 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:58.386573 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2d91a-ca88-49fe-bc72-c6adeb2abb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:58.389272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e79a8d-b2e7-4ee7-8cc7-11140ff7fa50 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:58.411014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-c2f2d91a-ca88-49fe-bc72-c6adeb2abb24 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:12:58.412346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-c2f2d91a-ca88-49fe-bc72-c6adeb2abb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:12:58.427635 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8aa45-e672-4fce-ae1d-570f880b4ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:58.428297 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8aa45-e672-4fce-ae1d-570f880b4ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:58.428569 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8aa45-e672-4fce-ae1d-570f880b4ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:58.428838 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee8aa45-e672-4fce-ae1d-570f880b4ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:58.430989 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8aa45-e672-4fce-ae1d-570f880b4ac4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:58.430989 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8aa45-e672-4fce-ae1d-570f880b4ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:58.431209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee8aa45-e672-4fce-ae1d-570f880b4ac4 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:58.441768 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 142/281] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:12:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:58.442789 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8aa45-e672-4fce-ae1d-570f880b4ac4 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:58.464836 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:58.509264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 140/282] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:12:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:12:58.510076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2d91a-ca88-49fe-bc72-c6adeb2abb24 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:58.591946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee8aa45-e672-4fce-ae1d-570f880b4ac4 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:58.622998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:58.637048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387164ea-7266-4e24-bf38-cf1e5197b534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:58.637990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387164ea-7266-4e24-bf38-cf1e5197b534 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:58.638407 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387164ea-7266-4e24-bf38-cf1e5197b534 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:58.638836 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387164ea-7266-4e24-bf38-cf1e5197b534 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:58.639691 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387164ea-7266-4e24-bf38-cf1e5197b534 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:58.640180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387164ea-7266-4e24-bf38-cf1e5197b534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:58.641819 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387164ea-7266-4e24-bf38-cf1e5197b534 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:58.652442 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 143/283] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:12:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:58.652803 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2d91a-ca88-49fe-bc72-c6adeb2abb24 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:58.653704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387164ea-7266-4e24-bf38-cf1e5197b534 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:58.692440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f1f86-97b0-4291-9a5d-db5a862d16b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:58.693315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f1f86-97b0-4291-9a5d-db5a862d16b0 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:58.693645 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f1f86-97b0-4291-9a5d-db5a862d16b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:58.694188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f1f86-97b0-4291-9a5d-db5a862d16b0 None admin] PATH_INFO: `/v3/users/1a486e8c82594beaa99454873a29e8e4` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:58.710842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f1f86-97b0-4291-9a5d-db5a862d16b0 None admin] RBAC: Authorizing `identity:get_user(user_id=1a486e8c82594beaa99454873a29e8e4)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:58.712631 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f1f86-97b0-4291-9a5d-db5a862d16b0 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:58.715256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 141/284] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:58 2022] GET /identity/v3/users/1a486e8c82594beaa99454873a29e8e4 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:58.716375 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f1f86-97b0-4291-9a5d-db5a862d16b0 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:58.749715 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:58.775506 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387164ea-7266-4e24-bf38-cf1e5197b534 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:58.817298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a515bc-aa62-460c-ae5d-f077ca1e9cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:58.817941 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a515bc-aa62-460c-ae5d-f077ca1e9cdb None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:58.818214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a515bc-aa62-460c-ae5d-f077ca1e9cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:58.818483 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a515bc-aa62-460c-ae5d-f077ca1e9cdb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:58.830784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a515bc-aa62-460c-ae5d-f077ca1e9cdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:12:58.837159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-a1a515bc-aa62-460c-ae5d-f077ca1e9cdb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:58.837159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:12:58.837159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:12:58.837159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:12:58.837159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:12:58.837159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:12:58.837159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:12:58.837159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:12:58.837159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:12:58.837159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:12:58.837159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:12:58.837159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:12:58.837159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:12:58.837159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:12:58.837159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:12:58.837159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:12:58.838060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:12:58.838060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:12:58.838060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:12:58.838060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:12:58.838060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:12:58.838060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:12:58.838060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:12:58.838060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:12:58.838060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:12:58.838060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:12:58.838060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:12:58.838060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:12:58.838060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:12:58.838060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:12:58.838060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:12:58.838060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:12:58.839743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:12:58.839743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:12:58.839743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:12:58.839743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:12:58.839743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:12:58.839743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:12:58.839743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:12:58.839743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:12:58.839743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:12:58.839743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:12:58.839743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:12:58.839743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:12:58.839743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 144/285] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:12:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:12:58.840704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a515bc-aa62-460c-ae5d-f077ca1e9cdb None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:58.847356 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f1f86-97b0-4291-9a5d-db5a862d16b0 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:58.886534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b37de3-9578-4f76-9221-7917ab34c1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:58.887844 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b37de3-9578-4f76-9221-7917ab34c1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:58.888369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b37de3-9578-4f76-9221-7917ab34c1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:58.888896 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b37de3-9578-4f76-9221-7917ab34c1e0 None admin] PATH_INFO: `/v3/users/aed21a5dcdc949b9a4e96ba85f023216` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:58.890410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b37de3-9578-4f76-9221-7917ab34c1e0 None admin] RBAC: Authorizing `identity:get_user(user_id=aed21a5dcdc949b9a4e96ba85f023216)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:58.892488 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b37de3-9578-4f76-9221-7917ab34c1e0 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:58.894672 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 142/286] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:58 2022] GET /identity/v3/users/aed21a5dcdc949b9a4e96ba85f023216 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:58.895887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b37de3-9578-4f76-9221-7917ab34c1e0 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:58.900465 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:58.978756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a515bc-aa62-460c-ae5d-f077ca1e9cdb None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:59.023452 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0b0cc-26a8-443e-b161-533bff7430a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:59.024290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0b0cc-26a8-443e-b161-533bff7430a1 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:59.024586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b37de3-9578-4f76-9221-7917ab34c1e0 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:59.024856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0b0cc-26a8-443e-b161-533bff7430a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:59.025281 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0b0cc-26a8-443e-b161-533bff7430a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:59.026284 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0b0cc-26a8-443e-b161-533bff7430a1 None admin] RBAC: Adding query filter params (role.id=246bbd6878334bba893bc6f5bf5caee0, user.id=1a486e8c82594beaa99454873a29e8e4, scope.system=all) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:59.026721 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0b0cc-26a8-443e-b161-533bff7430a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:59.028506 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0b0cc-26a8-443e-b161-533bff7430a1 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:59.031812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:59.035806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 145/287] 10.222.0.17 () {56 vars in 1308 bytes} [Sun Aug 7 19:12:58 2022] GET /identity/v3/role_assignments?user.id=1a486e8c82594beaa99454873a29e8e4&scope.system=all&role.id=246bbd6878334bba893bc6f5bf5caee0 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:59.036891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0b0cc-26a8-443e-b161-533bff7430a1 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:59.073472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2104e4-e1a8-48a1-8392-815eab2516ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:59.074820 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2104e4-e1a8-48a1-8392-815eab2516ad None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:59.075371 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2104e4-e1a8-48a1-8392-815eab2516ad None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:59.075872 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2104e4-e1a8-48a1-8392-815eab2516ad None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:59.076725 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2104e4-e1a8-48a1-8392-815eab2516ad None admin] RBAC: Adding query filter params (name=member) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:12:59.077169 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2104e4-e1a8-48a1-8392-815eab2516ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:59.080076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2104e4-e1a8-48a1-8392-815eab2516ad None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:59.091722 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 143/288] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:12:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:59.101105 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2104e4-e1a8-48a1-8392-815eab2516ad None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:59.164640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0b0cc-26a8-443e-b161-533bff7430a1 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:59.207514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94692c1-5fff-4642-a81d-e6db8892b1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:59.208390 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94692c1-5fff-4642-a81d-e6db8892b1fc None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:59.208767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94692c1-5fff-4642-a81d-e6db8892b1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:59.209139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94692c1-5fff-4642-a81d-e6db8892b1fc None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:59.210961 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94692c1-5fff-4642-a81d-e6db8892b1fc None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:59.212941 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94692c1-5fff-4642-a81d-e6db8892b1fc None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:59.214966 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 146/289] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:59 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:59.221658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94692c1-5fff-4642-a81d-e6db8892b1fc None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:59.273168 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:59.299891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:59.338928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2104e4-e1a8-48a1-8392-815eab2516ad None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:59.344518 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94692c1-5fff-4642-a81d-e6db8892b1fc None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:59.382008 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2193d3-4ee8-4a8b-ae0f-7bea2bc8a6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:59.382806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde49f9-de62-46d8-be44-148c84bf2c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:59.383225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2193d3-4ee8-4a8b-ae0f-7bea2bc8a6cc None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:59.383481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde49f9-de62-46d8-be44-148c84bf2c40 None admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:59.383762 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde49f9-de62-46d8-be44-148c84bf2c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:59.384067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde49f9-de62-46d8-be44-148c84bf2c40 None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469/users/aed21a5dcdc949b9a4e96ba85f023216/roles/31d708bdce024bbc8423f085d27b1776` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:59.384346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2193d3-4ee8-4a8b-ae0f-7bea2bc8a6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:59.384821 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2193d3-4ee8-4a8b-ae0f-7bea2bc8a6cc None admin] PATH_INFO: `/v3/users/07d34fb8652e423dbe23f77fa9277e1b` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:59.386064 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde49f9-de62-46d8-be44-148c84bf2c40 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad772dbadd1e498f9d58bf65502c4469, user_id=aed21a5dcdc949b9a4e96ba85f023216, role_id=31d708bdce024bbc8423f085d27b1776)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:59.386398 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2193d3-4ee8-4a8b-ae0f-7bea2bc8a6cc None admin] RBAC: Authorizing `identity:get_user(user_id=07d34fb8652e423dbe23f77fa9277e1b)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:59.387572 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde49f9-de62-46d8-be44-148c84bf2c40 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:59.388328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2193d3-4ee8-4a8b-ae0f-7bea2bc8a6cc None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:59.391349 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 144/290] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:12:59 2022] GET /identity/v3/users/07d34fb8652e423dbe23f77fa9277e1b => generated 297 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:12:59.396251 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2193d3-4ee8-4a8b-ae0f-7bea2bc8a6cc None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:59.415415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 147/291] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:12:59 2022] PUT /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469/users/aed21a5dcdc949b9a4e96ba85f023216/roles/31d708bdce024bbc8423f085d27b1776 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:12:59.556814 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2193d3-4ee8-4a8b-ae0f-7bea2bc8a6cc None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:59.604577 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:59.611136 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a834791-514b-4d5f-a209-78aee2f09f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:59.611766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a834791-514b-4d5f-a209-78aee2f09f9d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:59.612073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a834791-514b-4d5f-a209-78aee2f09f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:59.612349 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a834791-514b-4d5f-a209-78aee2f09f9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:59.698291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a834791-514b-4d5f-a209-78aee2f09f9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:12:59.699818 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a834791-514b-4d5f-a209-78aee2f09f9d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:12:59.819951 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 145/292] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:12:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 19:12:59.825802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde49f9-de62-46d8-be44-148c84bf2c40 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:12:59.940633 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde49f9-de62-46d8-be44-148c84bf2c40 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:12:59.981630 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:12:59.990460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcc268-c5ed-47f6-a68b-055fb4065386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:12:59.991425 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcc268-c5ed-47f6-a68b-055fb4065386 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:12:59.991739 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcc268-c5ed-47f6-a68b-055fb4065386 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:12:59.996655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcc268-c5ed-47f6-a68b-055fb4065386 None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:12:59.998569 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddcc268-c5ed-47f6-a68b-055fb4065386 None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:00.000578 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddcc268-c5ed-47f6-a68b-055fb4065386 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:00.002876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 148/293] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:12:59 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:00.016127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a834791-514b-4d5f-a209-78aee2f09f9d None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:00.071640 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:00.184481 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a834791-514b-4d5f-a209-78aee2f09f9d None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:00.224243 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9023e-48c4-4003-a656-c5ef9782494d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:00.225229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9023e-48c4-4003-a656-c5ef9782494d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:00.225699 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9023e-48c4-4003-a656-c5ef9782494d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:00.226179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9023e-48c4-4003-a656-c5ef9782494d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:00.227092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9023e-48c4-4003-a656-c5ef9782494d None admin] RBAC: Adding query filter params (role.id=246bbd6878334bba893bc6f5bf5caee0, scope.project.id=ad772dbadd1e498f9d58bf65502c4469, user.id=07d34fb8652e423dbe23f77fa9277e1b) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:00.227627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9023e-48c4-4003-a656-c5ef9782494d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:00.229289 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9023e-48c4-4003-a656-c5ef9782494d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:00.236227 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 146/294] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:13:00 2022] GET /identity/v3/role_assignments?user.id=07d34fb8652e423dbe23f77fa9277e1b&scope.project.id=ad772dbadd1e498f9d58bf65502c4469&role.id=246bbd6878334bba893bc6f5bf5caee0 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:00.845483 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e22f7-c7fc-476a-92e4-33f639597ede None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:00.845991 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e22f7-c7fc-476a-92e4-33f639597ede None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:00.846287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20e22f7-c7fc-476a-92e4-33f639597ede None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:00.848072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 149/295] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:00.854453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43694b0-9016-4fb0-9c94-de0d31daa3ae None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:00.854891 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43694b0-9016-4fb0-9c94-de0d31daa3ae None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:00.855078 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43694b0-9016-4fb0-9c94-de0d31daa3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:00.879183 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-b43694b0-9016-4fb0-9c94-de0d31daa3ae None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:00.880509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b43694b0-9016-4fb0-9c94-de0d31daa3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:00.936480 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c04395-5916-4f55-adec-0fc5b7dc8bb5 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:00.936949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c04395-5916-4f55-adec-0fc5b7dc8bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:00.937347 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c04395-5916-4f55-adec-0fc5b7dc8bb5 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:00.940254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 150/296] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:00.946875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a3524-2f28-4bbc-833d-a62da6ef93f0 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:00.947398 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a3524-2f28-4bbc-833d-a62da6ef93f0 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:00.947678 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a3524-2f28-4bbc-833d-a62da6ef93f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:00.964119 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 147/297] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:00.972198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-e11a3524-2f28-4bbc-833d-a62da6ef93f0 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:00.973648 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e11a3524-2f28-4bbc-833d-a62da6ef93f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:00.984939 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43694b0-9016-4fb0-9c94-de0d31daa3ae None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:01.023644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:01.069001 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 151/298] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:13:01.098848 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a3524-2f28-4bbc-833d-a62da6ef93f0 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:01.110039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43694b0-9016-4fb0-9c94-de0d31daa3ae None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:01.146098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225cb1e-313e-4104-9728-36d724982f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:01.146734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225cb1e-313e-4104-9728-36d724982f43 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:01.147117 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225cb1e-313e-4104-9728-36d724982f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:01.147386 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225cb1e-313e-4104-9728-36d724982f43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:01.148528 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225cb1e-313e-4104-9728-36d724982f43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:01.149983 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225cb1e-313e-4104-9728-36d724982f43 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:01.151695 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 148/299] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:01.158532 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225cb1e-313e-4104-9728-36d724982f43 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:01.229524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a3524-2f28-4bbc-833d-a62da6ef93f0 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:01.256901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:01.263855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225cb1e-313e-4104-9728-36d724982f43 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:01.265291 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:01.278795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d56d0-0a51-4c24-80ea-d3cf06665ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:01.280198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d56d0-0a51-4c24-80ea-d3cf06665ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:01.280749 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d56d0-0a51-4c24-80ea-d3cf06665ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:01.281191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d56d0-0a51-4c24-80ea-d3cf06665ac6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:01.299827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a73735-b37c-4c15-beca-159dede96835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:01.300713 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73735-b37c-4c15-beca-159dede96835 None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:01.301130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73735-b37c-4c15-beca-159dede96835 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:01.301561 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a73735-b37c-4c15-beca-159dede96835 None admin] PATH_INFO: `/v3/groups` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:01.302335 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a73735-b37c-4c15-beca-159dede96835 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:01.303776 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d56d0-0a51-4c24-80ea-d3cf06665ac6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:01.304093 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a73735-b37c-4c15-beca-159dede96835 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:01.314502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-7f4d56d0-0a51-4c24-80ea-d3cf06665ac6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:13:01.314502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:01.314502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:01.314502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:01.314502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:01.314502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:01.314502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:01.314502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:01.314502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:01.314502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:01.314502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:01.314502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:01.314502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:01.314502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:01.314502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:01.314502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:01.315468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:01.315468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:01.315468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:01.315468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:01.315468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:01.315468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:01.315468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:01.315468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:01.315468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:01.315468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:01.315468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:01.315468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:01.315468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:01.315468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:01.315468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:01.315468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:01.316287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:01.316287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:01.316287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:01.316287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:01.316287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:01.316287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:01.316287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:01.316287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:01.316287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:01.316287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:01.316287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:13:01.316287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:01.323903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 152/300] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:01.325126 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d56d0-0a51-4c24-80ea-d3cf06665ac6 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:01.337473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 149/301] 10.222.0.17 () {60 vars in 1134 bytes} [Sun Aug 7 19:13:01 2022] POST /identity/v3/groups => generated 229 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:13:01.457969 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d56d0-0a51-4c24-80ea-d3cf06665ac6 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:01.498217 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627fdfe5-aaa2-4cd9-af29-cf9fcada6062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:01.498825 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fdfe5-aaa2-4cd9-af29-cf9fcada6062 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:01.499104 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fdfe5-aaa2-4cd9-af29-cf9fcada6062 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:01.499374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fdfe5-aaa2-4cd9-af29-cf9fcada6062 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:01.500081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627fdfe5-aaa2-4cd9-af29-cf9fcada6062 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:01.500386 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627fdfe5-aaa2-4cd9-af29-cf9fcada6062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:01.501798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627fdfe5-aaa2-4cd9-af29-cf9fcada6062 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:01.512538 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 153/302] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:13:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:01.518564 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a73735-b37c-4c15-beca-159dede96835 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:01.567422 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1eb99-1e64-45a8-b1e0-1371015c2dd2 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:01.567824 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1eb99-1e64-45a8-b1e0-1371015c2dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:01.567907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c1eb99-1e64-45a8-b1e0-1371015c2dd2 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:01.575927 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 154/303] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:13:01.575927 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72e92b-9e42-4ce4-a78b-7bf75403d6c8 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:01.575927 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72e92b-9e42-4ce4-a78b-7bf75403d6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:01.575927 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72e92b-9e42-4ce4-a78b-7bf75403d6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:01.598679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-8d72e92b-9e42-4ce4-a78b-7bf75403d6c8 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:01.602988 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8d72e92b-9e42-4ce4-a78b-7bf75403d6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:01.641653 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a73735-b37c-4c15-beca-159dede96835 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:01.654862 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:01.680381 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc5b55-d10b-42ec-8c79-62f4f088b892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:01.681097 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc5b55-d10b-42ec-8c79-62f4f088b892 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:01.681373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc5b55-d10b-42ec-8c79-62f4f088b892 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:01.681701 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc5b55-d10b-42ec-8c79-62f4f088b892 None admin] PATH_INFO: `/v3/users/0cdf007a33024a60bab3d3f9dff39fe3` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:01.683168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 155/304] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:13:01.698238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc5b55-d10b-42ec-8c79-62f4f088b892 None admin] RBAC: Authorizing `identity:get_user(user_id=0cdf007a33024a60bab3d3f9dff39fe3)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:01.699798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc5b55-d10b-42ec-8c79-62f4f088b892 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:01.701904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 150/305] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:13:01 2022] GET /identity/v3/users/0cdf007a33024a60bab3d3f9dff39fe3 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:01.706837 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72e92b-9e42-4ce4-a78b-7bf75403d6c8 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:01.707696 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc5b55-d10b-42ec-8c79-62f4f088b892 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:01.727826 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:01.811027 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d72e92b-9e42-4ce4-a78b-7bf75403d6c8 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:01.824495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc5b55-d10b-42ec-8c79-62f4f088b892 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:01.846552 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251ac41-62ac-4652-9835-dcc7f79df77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:01.847884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251ac41-62ac-4652-9835-dcc7f79df77a None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:01.848943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251ac41-62ac-4652-9835-dcc7f79df77a None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:01.849444 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f251ac41-62ac-4652-9835-dcc7f79df77a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:01.859976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34410283-4c00-44ef-a068-4a92da6ba2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:01.860690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34410283-4c00-44ef-a068-4a92da6ba2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:01.860959 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34410283-4c00-44ef-a068-4a92da6ba2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:01.861223 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34410283-4c00-44ef-a068-4a92da6ba2b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:01.861958 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34410283-4c00-44ef-a068-4a92da6ba2b5 None admin] RBAC: Adding query filter params (role.id=31d708bdce024bbc8423f085d27b1776, user.id=0cdf007a33024a60bab3d3f9dff39fe3, scope.system=all) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:01.862267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34410283-4c00-44ef-a068-4a92da6ba2b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:01.863659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f251ac41-62ac-4652-9835-dcc7f79df77a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:01.863769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34410283-4c00-44ef-a068-4a92da6ba2b5 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:01.869324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-f251ac41-62ac-4652-9835-dcc7f79df77a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:13:01.869324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:01.869324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:01.869324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:01.869324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:01.869324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:01.869324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:01.869324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:01.869324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:01.869324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:01.869324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:01.869324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:01.869324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:01.869324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:01.869324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:01.869324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:01.870209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:01.870209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:01.870209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:01.870209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:01.870209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:01.870209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:01.870209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:01.870209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:01.870209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:01.870209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:01.870209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:01.870209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:01.870209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:01.870209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:01.870209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:01.870209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:01.871057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:01.871057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:01.871057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:01.871057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:01.871057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:01.871057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:01.871057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:01.871057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:01.871057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:01.871057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:01.871057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:13:01.871057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:01.871765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 151/306] 10.222.0.17 () {56 vars in 1308 bytes} [Sun Aug 7 19:13:01 2022] GET /identity/v3/role_assignments?user.id=0cdf007a33024a60bab3d3f9dff39fe3&scope.system=all&role.id=31d708bdce024bbc8423f085d27b1776 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:01.873591 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 156/307] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:01.885354 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34410283-4c00-44ef-a068-4a92da6ba2b5 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:01.982607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:02.009036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34410283-4c00-44ef-a068-4a92da6ba2b5 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:02.050719 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f91d3a-4e81-453d-809c-632037615540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:02.051351 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f91d3a-4e81-453d-809c-632037615540 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:02.051633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f91d3a-4e81-453d-809c-632037615540 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:02.051905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f91d3a-4e81-453d-809c-632037615540 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:02.052607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f91d3a-4e81-453d-809c-632037615540 None admin] RBAC: Adding query filter params (name=member) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:02.052909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f91d3a-4e81-453d-809c-632037615540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:02.054312 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f91d3a-4e81-453d-809c-632037615540 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:02.065151 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 152/308] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:13:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:02.070776 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251ac41-62ac-4652-9835-dcc7f79df77a None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:02.092872 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fed6cc-1009-442b-ad14-bfc10dcab660 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:02.093150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fed6cc-1009-442b-ad14-bfc10dcab660 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:02.093428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fed6cc-1009-442b-ad14-bfc10dcab660 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:02.095287 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 153/309] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:02.100561 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295b93d-cc74-4238-82dc-d9f76c3c8d5e None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:02.101109 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295b93d-cc74-4238-82dc-d9f76c3c8d5e None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:02.101563 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8295b93d-cc74-4238-82dc-d9f76c3c8d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:02.120886 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:02.124203 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-8295b93d-cc74-4238-82dc-d9f76c3c8d5e None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:02.125712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8295b93d-cc74-4238-82dc-d9f76c3c8d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:02.199109 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f251ac41-62ac-4652-9835-dcc7f79df77a None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:02.207131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 154/310] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:13:02.229720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8295b93d-cc74-4238-82dc-d9f76c3c8d5e None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:02.236708 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3bd73-f370-461c-b7cf-fc2fed106cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:02.237579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3bd73-f370-461c-b7cf-fc2fed106cdf None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:02.238049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3bd73-f370-461c-b7cf-fc2fed106cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:02.238567 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f3bd73-f370-461c-b7cf-fc2fed106cdf None admin] PATH_INFO: `/v3/users/1a486e8c82594beaa99454873a29e8e4` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:02.239952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f3bd73-f370-461c-b7cf-fc2fed106cdf None admin] RBAC: Authorizing `identity:get_user(user_id=1a486e8c82594beaa99454873a29e8e4)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:02.241734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f3bd73-f370-461c-b7cf-fc2fed106cdf None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:02.243811 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 157/311] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:13:02 2022] GET /identity/v3/users/1a486e8c82594beaa99454873a29e8e4 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:02.249864 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3bd73-f370-461c-b7cf-fc2fed106cdf None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:02.301054 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:02.334799 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8295b93d-cc74-4238-82dc-d9f76c3c8d5e None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:02.368330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ccb57-8212-4120-8770-6d4a18f20673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:02.368964 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ccb57-8212-4120-8770-6d4a18f20673 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:02.369237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ccb57-8212-4120-8770-6d4a18f20673 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:02.369507 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ccb57-8212-4120-8770-6d4a18f20673 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:02.369746 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f3bd73-f370-461c-b7cf-fc2fed106cdf None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:02.378349 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:02.380340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ccb57-8212-4120-8770-6d4a18f20673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:02.385271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-cc1ccb57-8212-4120-8770-6d4a18f20673 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:13:02.385271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:02.385271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:02.385271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:02.385271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:02.385271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:02.385271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:02.385271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:02.385271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:02.385271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:02.385271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:02.385271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:02.385271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:02.385271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:02.385271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:02.385271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:02.386175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:02.386175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:02.386175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:02.386175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:02.386175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:02.386175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:02.386175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:02.386175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:02.386175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:02.386175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:02.386175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:02.386175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:02.386175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:02.386175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:02.386175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:02.386175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:02.387098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:02.387098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:02.387098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:02.387098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:02.387098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:02.387098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:02.387098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:02.387098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:02.387098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:02.387098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:02.387098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:13:02.387098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:02.391619 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 155/312] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:02.392500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ccb57-8212-4120-8770-6d4a18f20673 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:02.413527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9d25d-f90f-43d7-9a3f-ea0776dd6059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:02.414580 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9d25d-f90f-43d7-9a3f-ea0776dd6059 None admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:02.415048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9d25d-f90f-43d7-9a3f-ea0776dd6059 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:02.415495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9d25d-f90f-43d7-9a3f-ea0776dd6059 None admin] PATH_INFO: `/v3/system/users/1a486e8c82594beaa99454873a29e8e4/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:02.417267 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9d25d-f90f-43d7-9a3f-ea0776dd6059 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1a486e8c82594beaa99454873a29e8e4, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:02.419058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9d25d-f90f-43d7-9a3f-ea0776dd6059 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:02.431727 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 158/313] 10.222.0.17 () {58 vars in 1334 bytes} [Sun Aug 7 19:13:02 2022] PUT /identity/v3/system/users/1a486e8c82594beaa99454873a29e8e4/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:13:02.524127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ccb57-8212-4120-8770-6d4a18f20673 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:02.555397 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:02.565618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37051a08-0b4d-4f67-86dc-2ebddacb8b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:02.566549 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37051a08-0b4d-4f67-86dc-2ebddacb8b08 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:02.567006 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37051a08-0b4d-4f67-86dc-2ebddacb8b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:02.567478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37051a08-0b4d-4f67-86dc-2ebddacb8b08 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:02.568363 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37051a08-0b4d-4f67-86dc-2ebddacb8b08 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:02.568846 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37051a08-0b4d-4f67-86dc-2ebddacb8b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:02.570500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37051a08-0b4d-4f67-86dc-2ebddacb8b08 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:02.584206 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 156/314] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:13:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:02.586946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9d25d-f90f-43d7-9a3f-ea0776dd6059 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:02.696788 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9d25d-f90f-43d7-9a3f-ea0776dd6059 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:02.731654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3affb85-0b55-4e41-af15-17e9ee4ae1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:02.732335 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3affb85-0b55-4e41-af15-17e9ee4ae1eb None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:02.732609 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3affb85-0b55-4e41-af15-17e9ee4ae1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:02.732882 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3affb85-0b55-4e41-af15-17e9ee4ae1eb None admin] PATH_INFO: `/v3/users/aed21a5dcdc949b9a4e96ba85f023216` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:02.734006 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3affb85-0b55-4e41-af15-17e9ee4ae1eb None admin] RBAC: Authorizing `identity:get_user(user_id=aed21a5dcdc949b9a4e96ba85f023216)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:02.735440 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3affb85-0b55-4e41-af15-17e9ee4ae1eb None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:02.737042 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 159/315] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:13:02 2022] GET /identity/v3/users/aed21a5dcdc949b9a4e96ba85f023216 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:02.742880 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37051a08-0b4d-4f67-86dc-2ebddacb8b08 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:02.852251 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37051a08-0b4d-4f67-86dc-2ebddacb8b08 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:02.886337 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f6a3c-8b6f-4bae-8523-e1cac31a6f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:02.887863 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f6a3c-8b6f-4bae-8523-e1cac31a6f31 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:02.888393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f6a3c-8b6f-4bae-8523-e1cac31a6f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:02.889227 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f6a3c-8b6f-4bae-8523-e1cac31a6f31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:02.949061 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:02.956690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f6a3c-8b6f-4bae-8523-e1cac31a6f31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:02.958118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489f6a3c-8b6f-4bae-8523-e1cac31a6f31 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:03.060825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 157/316] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:03.066349 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3affb85-0b55-4e41-af15-17e9ee4ae1eb None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:03.107653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:03.173802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3affb85-0b55-4e41-af15-17e9ee4ae1eb None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:03.209646 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67db89-ffa2-4c78-8965-3b1f04f4d32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:03.210536 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67db89-ffa2-4c78-8965-3b1f04f4d32d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:03.210952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67db89-ffa2-4c78-8965-3b1f04f4d32d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:03.211388 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67db89-ffa2-4c78-8965-3b1f04f4d32d None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:03.212672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67db89-ffa2-4c78-8965-3b1f04f4d32d None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:03.214261 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67db89-ffa2-4c78-8965-3b1f04f4d32d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:03.220096 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 160/317] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:13:03 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:03.222708 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f6a3c-8b6f-4bae-8523-e1cac31a6f31 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:03.315174 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:03.326531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f6a3c-8b6f-4bae-8523-e1cac31a6f31 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:03.360433 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e09327-7465-40a5-83b8-05ba391c054b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:03.360969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e09327-7465-40a5-83b8-05ba391c054b None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:03.361241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e09327-7465-40a5-83b8-05ba391c054b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:03.361515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e09327-7465-40a5-83b8-05ba391c054b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:03.362240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e09327-7465-40a5-83b8-05ba391c054b None admin] RBAC: Adding query filter params (role.id=31d708bdce024bbc8423f085d27b1776, scope.project.id=ad772dbadd1e498f9d58bf65502c4469, user.id=aed21a5dcdc949b9a4e96ba85f023216) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:03.362547 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e09327-7465-40a5-83b8-05ba391c054b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:03.364046 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e09327-7465-40a5-83b8-05ba391c054b None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:03.370038 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 158/318] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:13:03 2022] GET /identity/v3/role_assignments?user.id=aed21a5dcdc949b9a4e96ba85f023216&scope.project.id=ad772dbadd1e498f9d58bf65502c4469&role.id=31d708bdce024bbc8423f085d27b1776 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:03.563451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32127e24-a592-4826-adbf-02b5d8f8269b None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:03.563737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32127e24-a592-4826-adbf-02b5d8f8269b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:03.564041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32127e24-a592-4826-adbf-02b5d8f8269b None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:03.565579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 161/319] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:03.570799 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e2883-b588-4ed3-9c6d-2a5060174017 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:03.571091 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e2883-b588-4ed3-9c6d-2a5060174017 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:03.571374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e2883-b588-4ed3-9c6d-2a5060174017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:03.590546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-fe3e2883-b588-4ed3-9c6d-2a5060174017 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:03.591966 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-fe3e2883-b588-4ed3-9c6d-2a5060174017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:03.664568 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 159/320] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:03.688274 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32127e24-a592-4826-adbf-02b5d8f8269b None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:03.792989 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32127e24-a592-4826-adbf-02b5d8f8269b None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:03.800124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:03.829076 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894418b0-401e-46e3-96bb-caf0fb97baac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:03.829715 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894418b0-401e-46e3-96bb-caf0fb97baac None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:03.829994 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894418b0-401e-46e3-96bb-caf0fb97baac None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:03.830270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894418b0-401e-46e3-96bb-caf0fb97baac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:03.831370 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894418b0-401e-46e3-96bb-caf0fb97baac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:03.832870 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894418b0-401e-46e3-96bb-caf0fb97baac None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:03.834891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 162/321] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:03.841008 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e2883-b588-4ed3-9c6d-2a5060174017 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:03.925989 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:03.942940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e2883-b588-4ed3-9c6d-2a5060174017 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:03.973456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60977762-8dac-4ccf-815b-87252a6fb0f9 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:03.973731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60977762-8dac-4ccf-815b-87252a6fb0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:03.974020 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60977762-8dac-4ccf-815b-87252a6fb0f9 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:03.975505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 163/322] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:03.978217 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002f1ce-1df4-45a4-9224-606e88a917d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:03.978920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002f1ce-1df4-45a4-9224-606e88a917d1 None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:03.979430 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002f1ce-1df4-45a4-9224-606e88a917d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:03.979702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002f1ce-1df4-45a4-9224-606e88a917d1 None admin] PATH_INFO: `/v3/groups` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:03.980359 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002f1ce-1df4-45a4-9224-606e88a917d1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:03.980451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6368f5-691a-4757-aaeb-9f770e036632 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:03.980803 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6368f5-691a-4757-aaeb-9f770e036632 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:03.981145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6368f5-691a-4757-aaeb-9f770e036632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:03.981815 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002f1ce-1df4-45a4-9224-606e88a917d1 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:03.999971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-7c6368f5-691a-4757-aaeb-9f770e036632 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:04.000883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 160/323] 10.222.0.17 () {60 vars in 1134 bytes} [Sun Aug 7 19:13:03 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:13:04.001264 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-7c6368f5-691a-4757-aaeb-9f770e036632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:04.083589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 164/324] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:04.105719 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002f1ce-1df4-45a4-9224-606e88a917d1 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:04.219894 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002f1ce-1df4-45a4-9224-606e88a917d1 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:04.254319 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae55eb-cc58-415d-8b9a-a0fe55a7d3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:04.254959 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae55eb-cc58-415d-8b9a-a0fe55a7d3ff None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:04.255237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae55eb-cc58-415d-8b9a-a0fe55a7d3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:04.255509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae55eb-cc58-415d-8b9a-a0fe55a7d3ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:04.266375 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae55eb-cc58-415d-8b9a-a0fe55a7d3ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:04.271545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-76ae55eb-cc58-415d-8b9a-a0fe55a7d3ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:13:04.271545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:04.271545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:04.271545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:04.271545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:04.271545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:04.271545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:04.271545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:04.271545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:04.271545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:04.271545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:04.271545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:04.271545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:04.271545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:04.271545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:04.271545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:04.272457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:04.272457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:04.272457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:04.272457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:04.272457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:04.272457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:04.272457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:04.272457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:04.272457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:04.272457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:04.272457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:04.272457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:04.272457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:04.272457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:04.272457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:04.272457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:04.273246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:04.273246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:04.273246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:04.273246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:04.273246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:04.273246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:04.273246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:04.273246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:04.273246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:04.273246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:04.273246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:13:04.273246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:04.273246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 161/325] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:04.282419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6368f5-691a-4757-aaeb-9f770e036632 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:04.395983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6368f5-691a-4757-aaeb-9f770e036632 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:04.398690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:04.439713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792f06c-ae85-4d92-bd1f-3c9eca490862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:04.440368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792f06c-ae85-4d92-bd1f-3c9eca490862 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:04.440645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792f06c-ae85-4d92-bd1f-3c9eca490862 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:04.440912 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792f06c-ae85-4d92-bd1f-3c9eca490862 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:04.441578 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792f06c-ae85-4d92-bd1f-3c9eca490862 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:04.441874 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792f06c-ae85-4d92-bd1f-3c9eca490862 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:04.443286 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792f06c-ae85-4d92-bd1f-3c9eca490862 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:04.452694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 165/326] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:13:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:04.458097 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae55eb-cc58-415d-8b9a-a0fe55a7d3ff None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:04.462079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:04.533107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7342938-e02c-4998-972a-ecfa58ceb672 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:04.533283 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7342938-e02c-4998-972a-ecfa58ceb672 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:04.533556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7342938-e02c-4998-972a-ecfa58ceb672 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:04.535049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 166/327] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:13:04.540012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c7ce9-5ef1-4f0d-84c0-22e2aa69c15e None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:04.540402 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c7ce9-5ef1-4f0d-84c0-22e2aa69c15e None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:04.540769 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c7ce9-5ef1-4f0d-84c0-22e2aa69c15e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:04.561704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-855c7ce9-5ef1-4f0d-84c0-22e2aa69c15e None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:04.563492 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-855c7ce9-5ef1-4f0d-84c0-22e2aa69c15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:04.572948 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae55eb-cc58-415d-8b9a-a0fe55a7d3ff None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:04.611262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156c080-9475-49a8-9355-db1d7ae75e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:04.612083 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156c080-9475-49a8-9355-db1d7ae75e64 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:04.612454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156c080-9475-49a8-9355-db1d7ae75e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:04.612729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156c080-9475-49a8-9355-db1d7ae75e64 None admin] PATH_INFO: `/v3/users/0cdf007a33024a60bab3d3f9dff39fe3` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:04.616165 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156c080-9475-49a8-9355-db1d7ae75e64 None admin] RBAC: Authorizing `identity:get_user(user_id=0cdf007a33024a60bab3d3f9dff39fe3)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:04.617623 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156c080-9475-49a8-9355-db1d7ae75e64 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:04.619525 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 162/328] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:13:04 2022] GET /identity/v3/users/0cdf007a33024a60bab3d3f9dff39fe3 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:04.625622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156c080-9475-49a8-9355-db1d7ae75e64 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:04.652783 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 167/329] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:13:04.679228 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c7ce9-5ef1-4f0d-84c0-22e2aa69c15e None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:04.713975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:04.749407 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156c080-9475-49a8-9355-db1d7ae75e64 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:04.786125 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207eaee-85c5-473f-810e-28aef8918cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:04.786800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207eaee-85c5-473f-810e-28aef8918cb7 None admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:04.787081 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207eaee-85c5-473f-810e-28aef8918cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:04.787348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207eaee-85c5-473f-810e-28aef8918cb7 None admin] PATH_INFO: `/v3/system/users/0cdf007a33024a60bab3d3f9dff39fe3/roles/31d708bdce024bbc8423f085d27b1776` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:04.788787 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207eaee-85c5-473f-810e-28aef8918cb7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0cdf007a33024a60bab3d3f9dff39fe3, role_id=31d708bdce024bbc8423f085d27b1776)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:04.790284 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207eaee-85c5-473f-810e-28aef8918cb7 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:04.790770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:04.796910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c7ce9-5ef1-4f0d-84c0-22e2aa69c15e None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:04.802029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 163/330] 10.222.0.17 () {58 vars in 1334 bytes} [Sun Aug 7 19:13:04 2022] PUT /identity/v3/system/users/0cdf007a33024a60bab3d3f9dff39fe3/roles/31d708bdce024bbc8423f085d27b1776 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:13:04.838860 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfad4d-e0fd-47d6-9b16-d9bed61ca195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:04.839519 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfad4d-e0fd-47d6-9b16-d9bed61ca195 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:04.839791 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfad4d-e0fd-47d6-9b16-d9bed61ca195 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:04.840165 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfad4d-e0fd-47d6-9b16-d9bed61ca195 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:04.853928 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfad4d-e0fd-47d6-9b16-d9bed61ca195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:04.858875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-36bfad4d-e0fd-47d6-9b16-d9bed61ca195 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:13:04.858875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:04.858875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:04.858875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:04.858875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:04.858875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:04.858875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:04.858875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:04.858875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:04.858875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:04.858875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:04.858875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:04.858875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:04.858875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:04.858875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:04.858875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:04.859827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:04.859827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:04.859827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:04.859827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:04.859827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:04.859827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:04.859827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:04.859827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:04.859827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:04.859827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:04.859827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:04.859827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:04.859827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:04.859827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:04.859827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:04.859827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:04.860704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:04.860704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:04.860704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:04.860704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:04.860704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:04.860704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:04.860704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:04.860704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:04.860704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:04.860704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:04.860704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:13:04.860704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:04.860704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 168/331] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:04.866765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207eaee-85c5-473f-810e-28aef8918cb7 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:04.975580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207eaee-85c5-473f-810e-28aef8918cb7 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:05.012409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037e61e-9fe7-4e90-8992-6078f917f1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:05.012594 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037e61e-9fe7-4e90-8992-6078f917f1de None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:05.015742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037e61e-9fe7-4e90-8992-6078f917f1de None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:05.015742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037e61e-9fe7-4e90-8992-6078f917f1de None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:05.015742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037e61e-9fe7-4e90-8992-6078f917f1de None admin] RBAC: Adding query filter params (name=member) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:05.015742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037e61e-9fe7-4e90-8992-6078f917f1de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:05.015742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037e61e-9fe7-4e90-8992-6078f917f1de None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:05.024732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:05.025012 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 164/332] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:13:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:05.032404 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfad4d-e0fd-47d6-9b16-d9bed61ca195 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:05.158525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfad4d-e0fd-47d6-9b16-d9bed61ca195 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:05.206857 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b868e-49e7-42e0-83bc-538e71e246f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:05.207568 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b868e-49e7-42e0-83bc-538e71e246f7 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:05.207847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b868e-49e7-42e0-83bc-538e71e246f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:05.208152 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b868e-49e7-42e0-83bc-538e71e246f7 None admin] PATH_INFO: `/v3/users/1a486e8c82594beaa99454873a29e8e4` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:05.209337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b868e-49e7-42e0-83bc-538e71e246f7 None admin] RBAC: Authorizing `identity:get_user(user_id=1a486e8c82594beaa99454873a29e8e4)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:05.210911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b868e-49e7-42e0-83bc-538e71e246f7 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:05.212852 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 169/333] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:13:05 2022] GET /identity/v3/users/1a486e8c82594beaa99454873a29e8e4 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:05.223400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037e61e-9fe7-4e90-8992-6078f917f1de None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:05.343103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037e61e-9fe7-4e90-8992-6078f917f1de None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:05.383738 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33d2c9-4961-404e-9e1d-2e0e9fb96014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:05.384381 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33d2c9-4961-404e-9e1d-2e0e9fb96014 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:05.384658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33d2c9-4961-404e-9e1d-2e0e9fb96014 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:05.384927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33d2c9-4961-404e-9e1d-2e0e9fb96014 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:05.385657 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33d2c9-4961-404e-9e1d-2e0e9fb96014 None admin] RBAC: Adding query filter params (role.id=246bbd6878334bba893bc6f5bf5caee0, user.id=1a486e8c82594beaa99454873a29e8e4, scope.system=all) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:05.385963 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33d2c9-4961-404e-9e1d-2e0e9fb96014 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:05.387438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33d2c9-4961-404e-9e1d-2e0e9fb96014 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:05.404205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 165/334] 10.222.0.17 () {56 vars in 1308 bytes} [Sun Aug 7 19:13:05 2022] GET /identity/v3/role_assignments?user.id=1a486e8c82594beaa99454873a29e8e4&scope.system=all&role.id=246bbd6878334bba893bc6f5bf5caee0 => generated 510 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:06.741256 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3bcff-e53e-47bf-971c-ffad18d4ea5f None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:06.741781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3bcff-e53e-47bf-971c-ffad18d4ea5f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:06.741781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3bcff-e53e-47bf-971c-ffad18d4ea5f None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:06.743406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 170/335] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:06.749309 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c14295-6b13-4b6b-a185-e150af9dbd91 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:06.750211 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c14295-6b13-4b6b-a185-e150af9dbd91 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:06.750682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c14295-6b13-4b6b-a185-e150af9dbd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:06.774194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-c2c14295-6b13-4b6b-a185-e150af9dbd91 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:06.775765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-c2c14295-6b13-4b6b-a185-e150af9dbd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:06.843898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:06.874093 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 166/336] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:06.921668 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3bcff-e53e-47bf-971c-ffad18d4ea5f None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:06.959172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:07.063890 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3bcff-e53e-47bf-971c-ffad18d4ea5f None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:07.113817 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd4dba-85e6-4a31-aa94-1ba4799a105b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:07.114538 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd4dba-85e6-4a31-aa94-1ba4799a105b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:07.114817 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd4dba-85e6-4a31-aa94-1ba4799a105b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:07.115089 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd4dba-85e6-4a31-aa94-1ba4799a105b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:07.130821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd4dba-85e6-4a31-aa94-1ba4799a105b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:07.135563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-a8cd4dba-85e6-4a31-aa94-1ba4799a105b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:13:07.135563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:07.135563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:07.135563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:07.135563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:07.135563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:07.135563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:07.135563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:07.135563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:07.135563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:07.135563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:07.135563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:07.135563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:07.135563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:07.135563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:07.135563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:07.136459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:07.136459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:07.136459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:07.136459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:07.136459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:07.136459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:07.136459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:07.136459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:07.136459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:07.136459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:07.136459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:07.136459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:07.136459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:07.136459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:07.136459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:07.136459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:07.137382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:07.137382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:07.137382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:07.137382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:07.137382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:07.137382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:07.137382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:07.137382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:07.137382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:07.137382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:07.137382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:13:07.137382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:07.137382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 171/337] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:13:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:07.144942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c14295-6b13-4b6b-a185-e150af9dbd91 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:07.270866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c14295-6b13-4b6b-a185-e150af9dbd91 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:07.311227 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831cc065-7e2c-4a4e-a45f-1ab660299d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:07.312007 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cc065-7e2c-4a4e-a45f-1ab660299d74 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:07.312177 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cc065-7e2c-4a4e-a45f-1ab660299d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:07.312441 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831cc065-7e2c-4a4e-a45f-1ab660299d74 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:07.313122 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cc065-7e2c-4a4e-a45f-1ab660299d74 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:07.313429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cc065-7e2c-4a4e-a45f-1ab660299d74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:07.314883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831cc065-7e2c-4a4e-a45f-1ab660299d74 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:07.326731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 167/338] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:07.332260 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd4dba-85e6-4a31-aa94-1ba4799a105b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:07.432668 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:07.439078 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd4dba-85e6-4a31-aa94-1ba4799a105b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:07.480809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de8c12-f411-4acb-a0d4-247b8ab9ac52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:07.481430 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de8c12-f411-4acb-a0d4-247b8ab9ac52 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:07.481735 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de8c12-f411-4acb-a0d4-247b8ab9ac52 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:07.482084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de8c12-f411-4acb-a0d4-247b8ab9ac52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:07.530112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72749f36-98f3-493e-9757-0d471bfcf10e None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:07.530412 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72749f36-98f3-493e-9757-0d471bfcf10e None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:07.530585 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72749f36-98f3-493e-9757-0d471bfcf10e None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:07.531943 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 168/339] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:07.537203 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129027e-b812-451f-a4a1-9f8b734ae4d2 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:07.537490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129027e-b812-451f-a4a1-9f8b734ae4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:07.537765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129027e-b812-451f-a4a1-9f8b734ae4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:07.556018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de8c12-f411-4acb-a0d4-247b8ab9ac52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:07.557583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de8c12-f411-4acb-a0d4-247b8ab9ac52 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:07.580025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-3129027e-b812-451f-a4a1-9f8b734ae4d2 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:07.581325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-3129027e-b812-451f-a4a1-9f8b734ae4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:07.675361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:07.707285 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:07.709267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 169/340] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:13:07.740924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 172/341] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:07.746488 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129027e-b812-451f-a4a1-9f8b734ae4d2 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:07.749581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de8c12-f411-4acb-a0d4-247b8ab9ac52 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:07.893658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de8c12-f411-4acb-a0d4-247b8ab9ac52 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:07.930547 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638a31e-d2a8-4007-88d4-e2c6dc21e80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:07.931212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638a31e-d2a8-4007-88d4-e2c6dc21e80b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:07.931497 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638a31e-d2a8-4007-88d4-e2c6dc21e80b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:07.931773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638a31e-d2a8-4007-88d4-e2c6dc21e80b None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:07.933459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638a31e-d2a8-4007-88d4-e2c6dc21e80b None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:07.935146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638a31e-d2a8-4007-88d4-e2c6dc21e80b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:07.937200 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 173/342] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:13:07 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:07.944516 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638a31e-d2a8-4007-88d4-e2c6dc21e80b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:07.951921 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129027e-b812-451f-a4a1-9f8b734ae4d2 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:08.055956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:08.056299 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6415dc-5b90-4d33-bfbc-26263fce8a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:08.056976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6415dc-5b90-4d33-bfbc-26263fce8a63 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:08.057281 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6415dc-5b90-4d33-bfbc-26263fce8a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:08.057561 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6415dc-5b90-4d33-bfbc-26263fce8a63 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:08.082693 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6415dc-5b90-4d33-bfbc-26263fce8a63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:08.088909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-2a6415dc-5b90-4d33-bfbc-26263fce8a63 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:13:08.088909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:08.088909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:08.088909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:08.088909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:08.088909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:08.088909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:08.088909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:08.088909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:08.088909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:08.088909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:08.088909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:08.088909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:08.088909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:08.088909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:08.088909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:08.089761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:08.089761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:08.089761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:08.089761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:08.089761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:08.089761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:08.089761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:08.089761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:08.089761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:08.089761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:08.089761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:08.089761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:08.089761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:08.089761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:08.089761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:08.089761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:08.090589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:08.090589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:08.090589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:08.090589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:08.090589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:08.090589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:08.090589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:08.090589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:08.090589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:08.090589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:08.090589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:13:08.090589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:08.095683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 170/343] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:08.096696 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6415dc-5b90-4d33-bfbc-26263fce8a63 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:08.147806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638a31e-d2a8-4007-88d4-e2c6dc21e80b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:08.177145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:08.216336 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db090a5d-d790-48ef-b68c-a793aca403a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:08.217035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db090a5d-d790-48ef-b68c-a793aca403a8 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:08.217316 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db090a5d-d790-48ef-b68c-a793aca403a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:08.217591 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db090a5d-d790-48ef-b68c-a793aca403a8 None admin] PATH_INFO: `/v3/groups/654f30bbad9845df99d839be8f392c4e` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:08.231176 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db090a5d-d790-48ef-b68c-a793aca403a8 None admin] RBAC: Authorizing `identity:get_group(group_id=654f30bbad9845df99d839be8f392c4e)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:08.232720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db090a5d-d790-48ef-b68c-a793aca403a8 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:08.234766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 174/344] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Aug 7 19:13:07 2022] GET /identity/v3/groups/654f30bbad9845df99d839be8f392c4e => generated 226 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:08.236008 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa183ea8-612b-48a3-8de0-19b154f7bf7b None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:08.236303 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa183ea8-612b-48a3-8de0-19b154f7bf7b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:08.236640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa183ea8-612b-48a3-8de0-19b154f7bf7b None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:08.238188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 175/345] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:08.244752 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa183ea8-612b-48a3-8de0-19b154f7bf7b None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:08.269440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6415dc-5b90-4d33-bfbc-26263fce8a63 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:08.313516 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6acccd-afc0-4a8c-bc49-361ad62c01f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:08.314364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6acccd-afc0-4a8c-bc49-361ad62c01f6 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:08.314662 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6acccd-afc0-4a8c-bc49-361ad62c01f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:08.314938 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6acccd-afc0-4a8c-bc49-361ad62c01f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:08.316122 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6acccd-afc0-4a8c-bc49-361ad62c01f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:08.316432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6acccd-afc0-4a8c-bc49-361ad62c01f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:08.317894 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6acccd-afc0-4a8c-bc49-361ad62c01f6 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:08.328199 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 171/346] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:13:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:08.329450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6734d17-41e4-4f9a-bd98-0219a70a5e70 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:08.329738 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6734d17-41e4-4f9a-bd98-0219a70a5e70 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:08.330016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6734d17-41e4-4f9a-bd98-0219a70a5e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:08.352831 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-e6734d17-41e4-4f9a-bd98-0219a70a5e70 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:08.354098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e6734d17-41e4-4f9a-bd98-0219a70a5e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:08.402182 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa183ea8-612b-48a3-8de0-19b154f7bf7b None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:08.422665 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:08.445961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 172/347] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:08.447928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830fabd-0084-4429-bda9-9c6cfe961c18 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:08.448243 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830fabd-0084-4429-bda9-9c6cfe961c18 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:08.449094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830fabd-0084-4429-bda9-9c6cfe961c18 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:08.450542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 173/348] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:08.451114 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830fabd-0084-4429-bda9-9c6cfe961c18 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:08.452091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c171a0e-ff19-41a2-a925-72bf1bb96340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:08.452691 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c171a0e-ff19-41a2-a925-72bf1bb96340 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:08.452977 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c171a0e-ff19-41a2-a925-72bf1bb96340 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:08.453235 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c171a0e-ff19-41a2-a925-72bf1bb96340 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:08.453976 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c171a0e-ff19-41a2-a925-72bf1bb96340 None admin] RBAC: Adding query filter params (group.id=654f30bbad9845df99d839be8f392c4e, role.id=9a1c954216e34607a6788f44f07a593b, scope.project.id=cabd760392b6495e87b66b85b6a15ca0) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:08.454281 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c171a0e-ff19-41a2-a925-72bf1bb96340 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:08.455745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c171a0e-ff19-41a2-a925-72bf1bb96340 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:08.462290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 176/349] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Aug 7 19:13:08 2022] GET /identity/v3/role_assignments?group.id=654f30bbad9845df99d839be8f392c4e&scope.project.id=cabd760392b6495e87b66b85b6a15ca0&role.id=9a1c954216e34607a6788f44f07a593b => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:08.463534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828bead-bb02-4c6f-9d88-90b8ba6dea85 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:08.463812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828bead-bb02-4c6f-9d88-90b8ba6dea85 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:08.464262 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8828bead-bb02-4c6f-9d88-90b8ba6dea85 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:08.465601 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 177/350] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:08.466788 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebb524-ffe0-41e4-aa1e-8e2efc6fab4f None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:08.467077 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebb524-ffe0-41e4-aa1e-8e2efc6fab4f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:08.467345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebb524-ffe0-41e4-aa1e-8e2efc6fab4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:08.487006 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-b7ebb524-ffe0-41e4-aa1e-8e2efc6fab4f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:08.488376 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b7ebb524-ffe0-41e4-aa1e-8e2efc6fab4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:08.491490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:08.584694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 178/351] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:08.585984 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daea46b-f998-45a6-b33a-08437be914a3 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:08.586275 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daea46b-f998-45a6-b33a-08437be914a3 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:08.586547 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daea46b-f998-45a6-b33a-08437be914a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:08.602630 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830fabd-0084-4429-bda9-9c6cfe961c18 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:08.622757 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-1daea46b-f998-45a6-b33a-08437be914a3 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:08.625449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-1daea46b-f998-45a6-b33a-08437be914a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:08.650073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f5a3a-1c20-4c26-8353-7e50aca12085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:08.650710 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f5a3a-1c20-4c26-8353-7e50aca12085 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:08.650989 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f5a3a-1c20-4c26-8353-7e50aca12085 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:08.651261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7f5a3a-1c20-4c26-8353-7e50aca12085 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:08.716319 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 179/352] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:08.717591 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daea46b-f998-45a6-b33a-08437be914a3 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:08.731956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f5a3a-1c20-4c26-8353-7e50aca12085 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:08.733772 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7f5a3a-1c20-4c26-8353-7e50aca12085 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:08.767816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:08.770444 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:08.845979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daea46b-f998-45a6-b33a-08437be914a3 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:08.855693 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 174/353] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:08.856999 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f5a3a-1c20-4c26-8353-7e50aca12085 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:08.891840 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d1eae-5b6c-438c-a3c4-0544c7f5b800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:08.892880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d1eae-5b6c-438c-a3c4-0544c7f5b800 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:08.893407 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d1eae-5b6c-438c-a3c4-0544c7f5b800 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:08.893896 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d1eae-5b6c-438c-a3c4-0544c7f5b800 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:08.906107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d1eae-5b6c-438c-a3c4-0544c7f5b800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:08.911417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-9f9d1eae-5b6c-438c-a3c4-0544c7f5b800 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:13:08.911417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:08.911417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:08.911417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:08.911417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:08.911417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:08.911417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:08.911417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:08.911417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:08.911417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:08.911417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:08.911417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:08.911417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:08.911417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:08.911417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:08.911417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:08.912352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:08.912352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:08.912352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:08.912352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:08.912352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:08.912352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:08.912352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:08.912352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:08.912352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:08.912352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:08.912352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:08.912352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:08.912352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:08.912352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:08.912352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:08.912352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:08.913187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:08.913187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:08.913187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:08.913187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:08.913187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:08.913187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:08.913187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:08.913187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:08.913187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:08.913187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:08.913187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 19:13:08.913187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:08.914327 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 180/354] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:08.915147 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d1eae-5b6c-438c-a3c4-0544c7f5b800 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:08.993745 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7f5a3a-1c20-4c26-8353-7e50aca12085 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:09.033076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:09.040074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef51431-a9ac-4943-8b1d-bd54d9a1fda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:09.041650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef51431-a9ac-4943-8b1d-bd54d9a1fda6 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:09.041650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef51431-a9ac-4943-8b1d-bd54d9a1fda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:09.041650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef51431-a9ac-4943-8b1d-bd54d9a1fda6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:09.053583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:09.059121 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d1eae-5b6c-438c-a3c4-0544c7f5b800 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:09.060691 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef51431-a9ac-4943-8b1d-bd54d9a1fda6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:09.067173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-6ef51431-a9ac-4943-8b1d-bd54d9a1fda6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:13:09.067173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:09.067173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:09.067173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:09.067173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:09.067173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:09.067173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:09.067173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:09.067173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:09.067173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:09.067173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:09.067173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:09.067173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:09.067173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:09.067173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:09.067173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:09.068126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:09.068126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:09.068126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:09.068126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:09.068126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:09.068126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:09.068126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:09.068126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:09.068126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:09.068126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:09.068126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:09.068126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:09.068126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:09.068126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:09.068126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:09.068126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:09.068903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:09.068903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:09.068903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:09.068903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:09.068903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:09.068903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:09.068903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:09.068903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:09.068903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:09.068903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:09.068903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:13:09.068903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:09.068903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 175/355] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:09.069556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef51431-a9ac-4943-8b1d-bd54d9a1fda6 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:09.108229 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca070f-6168-4902-a131-103a63c9a124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:09.108874 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca070f-6168-4902-a131-103a63c9a124 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:09.109159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca070f-6168-4902-a131-103a63c9a124 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:09.109430 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca070f-6168-4902-a131-103a63c9a124 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:09.121463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca070f-6168-4902-a131-103a63c9a124 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:09.126225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-94ca070f-6168-4902-a131-103a63c9a124 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:13:09.126225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:09.126225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:09.126225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:09.126225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:09.126225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:09.126225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:09.126225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:09.126225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:09.126225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:09.126225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:09.126225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:09.126225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:09.126225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:09.126225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:09.126225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:09.127120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:09.127120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:09.127120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:09.127120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:09.127120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:09.127120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:09.127120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:09.127120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:09.127120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:09.127120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:09.127120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:09.127120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:09.127120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:09.127120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:09.127120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:09.127120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:09.128367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:09.128367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:09.128367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:09.128367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:09.128367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:09.128367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:09.128367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:09.128367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:09.128367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:09.128367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:09.128367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:13:09.128367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:09.128367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 181/356] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:13:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:09.130724 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca070f-6168-4902-a131-103a63c9a124 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:09.202622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef51431-a9ac-4943-8b1d-bd54d9a1fda6 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:09.250245 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88b331-1fb2-4e6c-8601-f488a8b377b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:09.250245 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88b331-1fb2-4e6c-8601-f488a8b377b7 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:09.250245 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88b331-1fb2-4e6c-8601-f488a8b377b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:09.250245 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88b331-1fb2-4e6c-8601-f488a8b377b7 None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:09.251579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88b331-1fb2-4e6c-8601-f488a8b377b7 None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:09.253749 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88b331-1fb2-4e6c-8601-f488a8b377b7 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:09.255523 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 176/357] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:13:09 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:09.257832 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21864ff-18f8-4f15-b074-e0c428ae831f None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:09.258119 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21864ff-18f8-4f15-b074-e0c428ae831f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:09.259371 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21864ff-18f8-4f15-b074-e0c428ae831f None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:09.259932 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 177/358] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:09.260535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21864ff-18f8-4f15-b074-e0c428ae831f None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:09.265850 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca070f-6168-4902-a131-103a63c9a124 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:09.317570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e4c54-df6e-424d-83ff-255e16a5a8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:09.318190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e4c54-df6e-424d-83ff-255e16a5a8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:09.318469 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e4c54-df6e-424d-83ff-255e16a5a8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:09.318736 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e4c54-df6e-424d-83ff-255e16a5a8c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:09.319412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e4c54-df6e-424d-83ff-255e16a5a8c4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:09.319713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e4c54-df6e-424d-83ff-255e16a5a8c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:09.321191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e4c54-df6e-424d-83ff-255e16a5a8c4 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:09.333211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 182/359] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:13:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:09.334996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e4c54-df6e-424d-83ff-255e16a5a8c4 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:09.337026 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:09.368069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:09.391734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21864ff-18f8-4f15-b074-e0c428ae831f None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:09.437329 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd594b-baa0-471a-bb40-b0c1331dad3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:09.438024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd594b-baa0-471a-bb40-b0c1331dad3c None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:09.438392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd594b-baa0-471a-bb40-b0c1331dad3c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:09.439851 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd594b-baa0-471a-bb40-b0c1331dad3c None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:09.439967 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd594b-baa0-471a-bb40-b0c1331dad3c None admin] RBAC: Adding query filter params (name=member) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:09.440310 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd594b-baa0-471a-bb40-b0c1331dad3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:09.442490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd594b-baa0-471a-bb40-b0c1331dad3c None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:09.457557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 178/360] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:13:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:09.458741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0f4b1-11b2-4046-8c6c-06b37f42d138 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:09.459028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0f4b1-11b2-4046-8c6c-06b37f42d138 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:09.459351 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0f4b1-11b2-4046-8c6c-06b37f42d138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:09.471422 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e4c54-df6e-424d-83ff-255e16a5a8c4 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:09.484974 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-43c0f4b1-11b2-4046-8c6c-06b37f42d138 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:09.486330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-43c0f4b1-11b2-4046-8c6c-06b37f42d138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:09.515169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b931176-5660-4cc1-9990-05e62594259c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:09.515708 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b931176-5660-4cc1-9990-05e62594259c None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:09.515984 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b931176-5660-4cc1-9990-05e62594259c None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:09.516323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b931176-5660-4cc1-9990-05e62594259c None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:09.517002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b931176-5660-4cc1-9990-05e62594259c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:09.517303 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b931176-5660-4cc1-9990-05e62594259c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:09.518699 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b931176-5660-4cc1-9990-05e62594259c None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:09.529534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 183/361] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:09.530757 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b931176-5660-4cc1-9990-05e62594259c None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:09.576330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 179/362] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:09.580036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0f4b1-11b2-4046-8c6c-06b37f42d138 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:09.642703 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:09.656548 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:09.660294 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b931176-5660-4cc1-9990-05e62594259c None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:09.701091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f73407-4638-45be-8d97-6542f7a13fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:09.702404 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f73407-4638-45be-8d97-6542f7a13fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:09.702920 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f73407-4638-45be-8d97-6542f7a13fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:09.703253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f73407-4638-45be-8d97-6542f7a13fc5 None admin] PATH_INFO: `/v3/groups/654f30bbad9845df99d839be8f392c4e` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:09.704501 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f73407-4638-45be-8d97-6542f7a13fc5 None admin] RBAC: Authorizing `identity:get_group(group_id=654f30bbad9845df99d839be8f392c4e)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:09.706563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f73407-4638-45be-8d97-6542f7a13fc5 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:09.709356 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 184/363] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Aug 7 19:13:09 2022] GET /identity/v3/groups/654f30bbad9845df99d839be8f392c4e => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:09.710151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f73407-4638-45be-8d97-6542f7a13fc5 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:09.728753 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0f4b1-11b2-4046-8c6c-06b37f42d138 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:09.769824 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07134b-426f-4fe6-b3ba-9814a55b0d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:09.770844 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07134b-426f-4fe6-b3ba-9814a55b0d38 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:09.771342 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07134b-426f-4fe6-b3ba-9814a55b0d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:09.771809 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07134b-426f-4fe6-b3ba-9814a55b0d38 None admin] PATH_INFO: `/v3/users/0cdf007a33024a60bab3d3f9dff39fe3` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:09.773320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e07134b-426f-4fe6-b3ba-9814a55b0d38 None admin] RBAC: Authorizing `identity:get_user(user_id=0cdf007a33024a60bab3d3f9dff39fe3)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:09.775005 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e07134b-426f-4fe6-b3ba-9814a55b0d38 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:09.777168 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 180/364] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:13:09 2022] GET /identity/v3/users/0cdf007a33024a60bab3d3f9dff39fe3 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:09.778307 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07134b-426f-4fe6-b3ba-9814a55b0d38 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:09.839666 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f73407-4638-45be-8d97-6542f7a13fc5 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:09.896122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044077c-a89b-4a00-a133-7fae7def7535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:09.897688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044077c-a89b-4a00-a133-7fae7def7535 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:09.898149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044077c-a89b-4a00-a133-7fae7def7535 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:09.898869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044077c-a89b-4a00-a133-7fae7def7535 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:09.902508 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07134b-426f-4fe6-b3ba-9814a55b0d38 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:09.927544 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:09.932350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:09.957287 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4dc149-59a7-47b9-a391-518d43e19def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:09.957917 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dc149-59a7-47b9-a391-518d43e19def None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:09.958199 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dc149-59a7-47b9-a391-518d43e19def None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:09.958473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4dc149-59a7-47b9-a391-518d43e19def None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:09.994738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044077c-a89b-4a00-a133-7fae7def7535 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:09.997142 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044077c-a89b-4a00-a133-7fae7def7535 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:10.040753 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4dc149-59a7-47b9-a391-518d43e19def None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:10.042407 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4dc149-59a7-47b9-a391-518d43e19def None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:10.146607 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 185/365] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:10.147723 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044077c-a89b-4a00-a133-7fae7def7535 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:10.199263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 181/366] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:10.203981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4dc149-59a7-47b9-a391-518d43e19def None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:10.226025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:10.226579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:10.301302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044077c-a89b-4a00-a133-7fae7def7535 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:10.330739 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4dc149-59a7-47b9-a391-518d43e19def None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:10.349557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf302fe-d6d5-4365-9405-e04b4ad8824a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:10.350208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf302fe-d6d5-4365-9405-e04b4ad8824a None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:10.350511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf302fe-d6d5-4365-9405-e04b4ad8824a None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:10.350752 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf302fe-d6d5-4365-9405-e04b4ad8824a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:10.367562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf302fe-d6d5-4365-9405-e04b4ad8824a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:10.372996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-ddf302fe-d6d5-4365-9405-e04b4ad8824a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:13:10.372996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:10.372996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:10.372996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:10.372996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:10.372996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:10.372996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:10.372996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:10.372996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:10.372996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:10.372996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:10.372996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:10.372996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:10.372996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:10.372996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:10.372996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:10.373854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:10.373854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:10.373854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:10.373854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:10.373854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:10.373854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:10.373854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:10.373854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:10.373854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:10.373854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:10.373854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:10.373854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:10.373854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:10.373854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:10.373854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:10.373854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:10.374631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:10.374631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:10.374631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:10.374631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:10.374631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:10.374631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:10.374631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:10.374631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:10.374631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:10.374631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:10.374631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:13:10.374631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:10.375225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f0eaf-16e4-490a-aa54-bb3d0ef6d6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:10.375225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f0eaf-16e4-490a-aa54-bb3d0ef6d6cf None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:10.375225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f0eaf-16e4-490a-aa54-bb3d0ef6d6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:10.375225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f0eaf-16e4-490a-aa54-bb3d0ef6d6cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:10.375470 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f0eaf-16e4-490a-aa54-bb3d0ef6d6cf None admin] RBAC: Adding query filter params (group.id=654f30bbad9845df99d839be8f392c4e, role.id=246bbd6878334bba893bc6f5bf5caee0, scope.project.id=cabd760392b6495e87b66b85b6a15ca0) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:10.375470 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f0eaf-16e4-490a-aa54-bb3d0ef6d6cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:10.376706 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f0eaf-16e4-490a-aa54-bb3d0ef6d6cf None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:10.377537 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 186/367] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:13:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:10.378517 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf302fe-d6d5-4365-9405-e04b4ad8824a None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:10.383237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 182/368] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Aug 7 19:13:10 2022] GET /identity/v3/role_assignments?group.id=654f30bbad9845df99d839be8f392c4e&scope.project.id=cabd760392b6495e87b66b85b6a15ca0&role.id=246bbd6878334bba893bc6f5bf5caee0 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:10.384068 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f0eaf-16e4-490a-aa54-bb3d0ef6d6cf None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:10.504564 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:10.529990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f0eaf-16e4-490a-aa54-bb3d0ef6d6cf None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:10.565969 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf302fe-d6d5-4365-9405-e04b4ad8824a None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:10.590138 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:10.593445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2087c38-b630-4e17-96a8-ee1168f91576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:10.594687 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2087c38-b630-4e17-96a8-ee1168f91576 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:10.595320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2087c38-b630-4e17-96a8-ee1168f91576 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:10.595953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2087c38-b630-4e17-96a8-ee1168f91576 None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:10.598184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2087c38-b630-4e17-96a8-ee1168f91576 None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:10.599813 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2087c38-b630-4e17-96a8-ee1168f91576 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:10.601971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 183/369] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:13:10 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:10.602928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2087c38-b630-4e17-96a8-ee1168f91576 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:10.611695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91b5f1-3294-42c4-bf1c-b447107890bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:10.612624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91b5f1-3294-42c4-bf1c-b447107890bb None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:10.613073 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91b5f1-3294-42c4-bf1c-b447107890bb None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:10.613490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91b5f1-3294-42c4-bf1c-b447107890bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:10.615052 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91b5f1-3294-42c4-bf1c-b447107890bb None admin] RBAC: Adding query filter params (role.id=31d708bdce024bbc8423f085d27b1776, user.id=0cdf007a33024a60bab3d3f9dff39fe3, scope.system=all) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:10.615527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91b5f1-3294-42c4-bf1c-b447107890bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:10.618107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91b5f1-3294-42c4-bf1c-b447107890bb None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:10.635293 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 187/370] 10.222.0.17 () {56 vars in 1308 bytes} [Sun Aug 7 19:13:10 2022] GET /identity/v3/role_assignments?user.id=0cdf007a33024a60bab3d3f9dff39fe3&scope.system=all&role.id=31d708bdce024bbc8423f085d27b1776 => generated 510 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:10.636610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91b5f1-3294-42c4-bf1c-b447107890bb None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:10.783858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2087c38-b630-4e17-96a8-ee1168f91576 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:10.812226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91b5f1-3294-42c4-bf1c-b447107890bb None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:10.827641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605f7bf-70e4-43f8-a535-dc14c13b3575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:10.828576 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605f7bf-70e4-43f8-a535-dc14c13b3575 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:10.829062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605f7bf-70e4-43f8-a535-dc14c13b3575 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:10.829480 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605f7bf-70e4-43f8-a535-dc14c13b3575 None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:10.831199 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0605f7bf-70e4-43f8-a535-dc14c13b3575 None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:10.832822 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0605f7bf-70e4-43f8-a535-dc14c13b3575 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:10.834668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 184/371] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:13:10 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:10.835435 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605f7bf-70e4-43f8-a535-dc14c13b3575 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:10.857203 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf3d6ff-29e2-4a9e-968a-73efe9262147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:10.858397 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:10.859620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3d6ff-29e2-4a9e-968a-73efe9262147 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:10.860218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3d6ff-29e2-4a9e-968a-73efe9262147 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:10.860218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf3d6ff-29e2-4a9e-968a-73efe9262147 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:10.860998 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3d6ff-29e2-4a9e-968a-73efe9262147 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:10.861181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3d6ff-29e2-4a9e-968a-73efe9262147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:10.863086 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf3d6ff-29e2-4a9e-968a-73efe9262147 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:10.877450 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 188/372] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:10.878967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2bc4f-ddd2-4616-84c2-a288c83bfb58 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:10.880103 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2bc4f-ddd2-4616-84c2-a288c83bfb58 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:10.880400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2bc4f-ddd2-4616-84c2-a288c83bfb58 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:10.881665 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 189/373] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:10.885787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2bc4f-ddd2-4616-84c2-a288c83bfb58 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:10.934217 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:10.968652 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605f7bf-70e4-43f8-a535-dc14c13b3575 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:11.016589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901eec4-7cdd-40f8-af0e-c47c472ca391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:11.017588 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901eec4-7cdd-40f8-af0e-c47c472ca391 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:11.018825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901eec4-7cdd-40f8-af0e-c47c472ca391 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:11.018825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8901eec4-7cdd-40f8-af0e-c47c472ca391 None admin] PATH_INFO: `/v3/groups/654f30bbad9845df99d839be8f392c4e` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:11.019389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901eec4-7cdd-40f8-af0e-c47c472ca391 None admin] RBAC: Authorizing `identity:get_group(group_id=654f30bbad9845df99d839be8f392c4e)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:11.020955 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8901eec4-7cdd-40f8-af0e-c47c472ca391 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:11.022734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 185/374] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Aug 7 19:13:10 2022] GET /identity/v3/groups/654f30bbad9845df99d839be8f392c4e => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:11.023300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901eec4-7cdd-40f8-af0e-c47c472ca391 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:11.048720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2bc4f-ddd2-4616-84c2-a288c83bfb58 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:11.101780 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abd227-696a-4e59-97e7-86d299d49dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:11.102349 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abd227-696a-4e59-97e7-86d299d49dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:11.102622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abd227-696a-4e59-97e7-86d299d49dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:11.102889 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abd227-696a-4e59-97e7-86d299d49dd5 None admin] PATH_INFO: `/v3/groups/654f30bbad9845df99d839be8f392c4e` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:11.104010 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68abd227-696a-4e59-97e7-86d299d49dd5 None admin] RBAC: Authorizing `identity:get_group(group_id=654f30bbad9845df99d839be8f392c4e)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:11.105532 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68abd227-696a-4e59-97e7-86d299d49dd5 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:11.107220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 190/375] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Aug 7 19:13:10 2022] GET /identity/v3/groups/654f30bbad9845df99d839be8f392c4e => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:11.108364 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb86c39-15ef-48a5-85f1-c9369fbceb84 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:11.108642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb86c39-15ef-48a5-85f1-c9369fbceb84 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:11.109021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb86c39-15ef-48a5-85f1-c9369fbceb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:11.140682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:11.141235 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-bbb86c39-15ef-48a5-85f1-c9369fbceb84 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:11.143340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-bbb86c39-15ef-48a5-85f1-c9369fbceb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:11.163669 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8901eec4-7cdd-40f8-af0e-c47c472ca391 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:11.235522 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fa40f-68b6-41b9-bb59-58aafafdc681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:11.236879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fa40f-68b6-41b9-bb59-58aafafdc681 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:11.236879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fa40f-68b6-41b9-bb59-58aafafdc681 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:11.236879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3fa40f-68b6-41b9-bb59-58aafafdc681 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:11.271965 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 191/376] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:11.272908 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb86c39-15ef-48a5-85f1-c9369fbceb84 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:11.326741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:11.354773 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fa40f-68b6-41b9-bb59-58aafafdc681 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:11.356270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3fa40f-68b6-41b9-bb59-58aafafdc681 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:11.427426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb86c39-15ef-48a5-85f1-c9369fbceb84 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:11.477159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a475f8c5-79f1-4c6d-af0e-674624230f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:11.477766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475f8c5-79f1-4c6d-af0e-674624230f31 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:11.478102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475f8c5-79f1-4c6d-af0e-674624230f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:11.478386 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a475f8c5-79f1-4c6d-af0e-674624230f31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:11.479111 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a475f8c5-79f1-4c6d-af0e-674624230f31 None admin] RBAC: Adding query filter params (group.id=654f30bbad9845df99d839be8f392c4e, role.id=246bbd6878334bba893bc6f5bf5caee0, scope.project.id=ad772dbadd1e498f9d58bf65502c4469) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:11.479424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a475f8c5-79f1-4c6d-af0e-674624230f31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:11.480943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a475f8c5-79f1-4c6d-af0e-674624230f31 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:11.486252 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:11.488029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 192/377] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Aug 7 19:13:11 2022] GET /identity/v3/role_assignments?group.id=654f30bbad9845df99d839be8f392c4e&scope.project.id=ad772dbadd1e498f9d58bf65502c4469&role.id=246bbd6878334bba893bc6f5bf5caee0 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:11.492254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a475f8c5-79f1-4c6d-af0e-674624230f31 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:11.505309 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 186/378] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:11.506208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fa40f-68b6-41b9-bb59-58aafafdc681 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:11.702292 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3fa40f-68b6-41b9-bb59-58aafafdc681 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:11.720553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:11.733027 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a475f8c5-79f1-4c6d-af0e-674624230f31 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:11.746729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3546f-876f-402d-abd3-eb86d0b98235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:11.747860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3546f-876f-402d-abd3-eb86d0b98235 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:11.748261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3546f-876f-402d-abd3-eb86d0b98235 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:11.748611 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3546f-876f-402d-abd3-eb86d0b98235 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:11.774700 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3546f-876f-402d-abd3-eb86d0b98235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:11.780971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-0eb3546f-876f-402d-abd3-eb86d0b98235 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:13:11.780971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:11.780971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:11.780971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:11.780971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:11.780971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:11.780971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:11.780971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:11.780971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:11.780971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:11.780971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:11.780971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:11.780971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:11.780971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:11.780971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:11.780971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:11.781956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:11.781956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:11.781956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:11.781956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:11.781956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:11.781956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:11.781956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:11.781956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:11.781956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:11.781956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:11.781956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:11.781956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:11.781956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:11.781956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:11.781956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:11.781956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:11.782797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:11.782797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:11.782797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:11.782797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:11.782797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:11.782797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:11.782797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:11.782797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:11.782797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:11.782797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:11.782797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:13:11.782797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:11.785283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 187/379] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:13:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:11.786225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3546f-876f-402d-abd3-eb86d0b98235 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:11.787978 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d260fd2-f654-4cdf-8fe9-c18ee947b288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:11.788878 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d260fd2-f654-4cdf-8fe9-c18ee947b288 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:11.789355 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d260fd2-f654-4cdf-8fe9-c18ee947b288 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:11.789972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d260fd2-f654-4cdf-8fe9-c18ee947b288 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:11.790953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d260fd2-f654-4cdf-8fe9-c18ee947b288 None admin] RBAC: Adding query filter params (group.id=654f30bbad9845df99d839be8f392c4e, role.id=9a1c954216e34607a6788f44f07a593b, scope.project.id=ad772dbadd1e498f9d58bf65502c4469) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:11.791466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d260fd2-f654-4cdf-8fe9-c18ee947b288 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:11.793371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d260fd2-f654-4cdf-8fe9-c18ee947b288 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:11.801107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 193/380] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Aug 7 19:13:11 2022] GET /identity/v3/role_assignments?group.id=654f30bbad9845df99d839be8f392c4e&scope.project.id=ad772dbadd1e498f9d58bf65502c4469&role.id=9a1c954216e34607a6788f44f07a593b => generated 260 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:11.803057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d260fd2-f654-4cdf-8fe9-c18ee947b288 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:11.916835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d260fd2-f654-4cdf-8fe9-c18ee947b288 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:11.925410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:11.957758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e81d4-0044-498c-a629-42a96ce36f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:11.958841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e81d4-0044-498c-a629-42a96ce36f36 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:11.959322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e81d4-0044-498c-a629-42a96ce36f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:11.959821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e81d4-0044-498c-a629-42a96ce36f36 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:11.960816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e81d4-0044-498c-a629-42a96ce36f36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:11.961352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e81d4-0044-498c-a629-42a96ce36f36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:11.963190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e81d4-0044-498c-a629-42a96ce36f36 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:11.972981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 194/381] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:11.981292 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e81d4-0044-498c-a629-42a96ce36f36 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:12.011360 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3546f-876f-402d-abd3-eb86d0b98235 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:12.056638 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe8db1-3214-42ab-94fd-6c6e6667f990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:12.058048 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe8db1-3214-42ab-94fd-6c6e6667f990 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:12.058342 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe8db1-3214-42ab-94fd-6c6e6667f990 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:12.058697 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe8db1-3214-42ab-94fd-6c6e6667f990 None admin] PATH_INFO: `/v3/projects/6c00a4064cc8477492c9b8b2e5412197` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:12.059861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe8db1-3214-42ab-94fd-6c6e6667f990 None admin] RBAC: Authorizing `identity:get_project(project_id=6c00a4064cc8477492c9b8b2e5412197)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:12.061344 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe8db1-3214-42ab-94fd-6c6e6667f990 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:12.062912 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:12.065053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 188/382] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:13:11 2022] GET /identity/v3/projects/6c00a4064cc8477492c9b8b2e5412197 => generated 344 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:12.078408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe8db1-3214-42ab-94fd-6c6e6667f990 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:12.232194 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e81d4-0044-498c-a629-42a96ce36f36 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:12.271001 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe8db1-3214-42ab-94fd-6c6e6667f990 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:12.300108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75d1ee-7983-4010-9577-ad6d84048aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:12.300799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75d1ee-7983-4010-9577-ad6d84048aed None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:12.301074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75d1ee-7983-4010-9577-ad6d84048aed None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:12.301341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75d1ee-7983-4010-9577-ad6d84048aed None admin] PATH_INFO: `/v3/groups/654f30bbad9845df99d839be8f392c4e` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:12.302635 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75d1ee-7983-4010-9577-ad6d84048aed None admin] RBAC: Authorizing `identity:get_group(group_id=654f30bbad9845df99d839be8f392c4e)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:12.303125 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:12.304107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75d1ee-7983-4010-9577-ad6d84048aed None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:12.309083 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 195/383] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Aug 7 19:13:11 2022] GET /identity/v3/groups/654f30bbad9845df99d839be8f392c4e => generated 226 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:12.315574 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75d1ee-7983-4010-9577-ad6d84048aed None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:12.323970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e93be-8e5d-4a39-9d5e-32d559973a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:12.324700 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e93be-8e5d-4a39-9d5e-32d559973a22 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:12.325020 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e93be-8e5d-4a39-9d5e-32d559973a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:12.327339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e93be-8e5d-4a39-9d5e-32d559973a22 None admin] PATH_INFO: `/v3/groups/62f4263724184b8282a082eacc7ac6ac` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:12.341629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e93be-8e5d-4a39-9d5e-32d559973a22 None admin] RBAC: Authorizing `identity:get_group(group_id=62f4263724184b8282a082eacc7ac6ac)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:12.343161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62e93be-8e5d-4a39-9d5e-32d559973a22 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:12.352365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 189/384] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Aug 7 19:13:12 2022] GET /identity/v3/groups/62f4263724184b8282a082eacc7ac6ac => generated 229 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:12.353309 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e93be-8e5d-4a39-9d5e-32d559973a22 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:12.506421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:12.522847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75d1ee-7983-4010-9577-ad6d84048aed None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:12.570937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f708c-d54d-4fe9-a635-e624c4fdb2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:12.572018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f708c-d54d-4fe9-a635-e624c4fdb2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:12.572643 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f708c-d54d-4fe9-a635-e624c4fdb2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:12.573141 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f708c-d54d-4fe9-a635-e624c4fdb2d8 None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:12.575508 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f708c-d54d-4fe9-a635-e624c4fdb2d8 None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:12.578634 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f708c-d54d-4fe9-a635-e624c4fdb2d8 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:12.581920 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 196/385] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:13:12 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:12.590700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f708c-d54d-4fe9-a635-e624c4fdb2d8 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:12.636112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62e93be-8e5d-4a39-9d5e-32d559973a22 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:12.727294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa9de3-b0e3-4d76-ac3d-1784671c5116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:12.727931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa9de3-b0e3-4d76-ac3d-1784671c5116 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:12.729899 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa9de3-b0e3-4d76-ac3d-1784671c5116 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:12.730201 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa9de3-b0e3-4d76-ac3d-1784671c5116 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:12.731441 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa9de3-b0e3-4d76-ac3d-1784671c5116 None admin] RBAC: Adding query filter params (group.id=62f4263724184b8282a082eacc7ac6ac, role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=6c00a4064cc8477492c9b8b2e5412197) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:12.731951 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa9de3-b0e3-4d76-ac3d-1784671c5116 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:12.733942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa9de3-b0e3-4d76-ac3d-1784671c5116 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:12.742781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f708c-d54d-4fe9-a635-e624c4fdb2d8 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:12.746408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 190/386] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Aug 7 19:13:12 2022] GET /identity/v3/role_assignments?group.id=62f4263724184b8282a082eacc7ac6ac&scope.project.id=6c00a4064cc8477492c9b8b2e5412197&role.id=4da42f08d107448fb877b42b61bc74ea => generated 260 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:12.748076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642dd3f-c59f-4811-8575-243e71108f9a None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:12.748569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642dd3f-c59f-4811-8575-243e71108f9a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:12.749066 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642dd3f-c59f-4811-8575-243e71108f9a None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:12.750659 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 191/387] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:12.757154 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5df018-afa7-40f9-a4e0-f6c6ff969e22 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:12.757790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5df018-afa7-40f9-a4e0-f6c6ff969e22 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:12.758314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5df018-afa7-40f9-a4e0-f6c6ff969e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:12.778107 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-1b5df018-afa7-40f9-a4e0-f6c6ff969e22 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:12.779765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1b5df018-afa7-40f9-a4e0-f6c6ff969e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:12.789812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17876608-3c78-4e0f-8e2c-696d9e8f2382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:12.790516 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17876608-3c78-4e0f-8e2c-696d9e8f2382 None admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:12.790794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17876608-3c78-4e0f-8e2c-696d9e8f2382 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:12.791067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17876608-3c78-4e0f-8e2c-696d9e8f2382 None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0/groups/654f30bbad9845df99d839be8f392c4e/roles/9a1c954216e34607a6788f44f07a593b` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:12.798114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17876608-3c78-4e0f-8e2c-696d9e8f2382 None admin] RBAC: Authorizing `identity:create_grant(project_id=cabd760392b6495e87b66b85b6a15ca0, group_id=654f30bbad9845df99d839be8f392c4e, role_id=9a1c954216e34607a6788f44f07a593b)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:12.799625 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17876608-3c78-4e0f-8e2c-696d9e8f2382 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:12.826233 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 197/388] 10.222.0.17 () {58 vars in 1442 bytes} [Sun Aug 7 19:13:12 2022] PUT /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0/groups/654f30bbad9845df99d839be8f392c4e/roles/9a1c954216e34607a6788f44f07a593b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:13:12.837827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:12.886552 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 192/389] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:13:12.936353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17876608-3c78-4e0f-8e2c-696d9e8f2382 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:12.962416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:13.067551 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17876608-3c78-4e0f-8e2c-696d9e8f2382 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:13.116598 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02749cb3-de59-4cb5-9863-6567184e7e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:13.117302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02749cb3-de59-4cb5-9863-6567184e7e2d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:13.117581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02749cb3-de59-4cb5-9863-6567184e7e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:13.117855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02749cb3-de59-4cb5-9863-6567184e7e2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:13.131330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02749cb3-de59-4cb5-9863-6567184e7e2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:13.136865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-02749cb3-de59-4cb5-9863-6567184e7e2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:13:13.136865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:13.136865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:13.136865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:13.136865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:13.136865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:13.136865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:13.136865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:13.136865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:13.136865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:13.136865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:13.136865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:13.136865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:13.136865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:13.136865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:13.136865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:13.137798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:13.137798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:13.137798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:13.137798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:13.137798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:13.137798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:13.137798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:13.137798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:13.137798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:13.137798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:13.137798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:13.137798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:13.137798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:13.137798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:13.137798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:13.137798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:13.138869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:13.138869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:13.138869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:13.138869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:13.138869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:13.138869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:13.138869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:13.138869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:13.138869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:13.138869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:13.138869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:13:13.138869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:13.138869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 198/390] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:13.146313 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5df018-afa7-40f9-a4e0-f6c6ff969e22 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:13.300292 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5df018-afa7-40f9-a4e0-f6c6ff969e22 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:13.345658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44316b-784d-4c04-9f13-1143aabbdc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:13.346301 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44316b-784d-4c04-9f13-1143aabbdc91 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:13.346583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44316b-784d-4c04-9f13-1143aabbdc91 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:13.346854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44316b-784d-4c04-9f13-1143aabbdc91 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:13.347535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44316b-784d-4c04-9f13-1143aabbdc91 None admin] RBAC: Adding query filter params (name=member) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:13.347838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44316b-784d-4c04-9f13-1143aabbdc91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:13.349279 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44316b-784d-4c04-9f13-1143aabbdc91 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:13.364367 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 193/391] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:13:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:13.370510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02749cb3-de59-4cb5-9863-6567184e7e2d None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:13.530190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:13.564039 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02749cb3-de59-4cb5-9863-6567184e7e2d None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:13.606755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db1acb-751e-4d58-ae7f-f80a6f7cfdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:13.607463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db1acb-751e-4d58-ae7f-f80a6f7cfdcd None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:13.607745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db1acb-751e-4d58-ae7f-f80a6f7cfdcd None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:13.608044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db1acb-751e-4d58-ae7f-f80a6f7cfdcd None admin] PATH_INFO: `/v3/groups/654f30bbad9845df99d839be8f392c4e` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:13.609301 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db1acb-751e-4d58-ae7f-f80a6f7cfdcd None admin] RBAC: Authorizing `identity:get_group(group_id=654f30bbad9845df99d839be8f392c4e)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:13.610775 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db1acb-751e-4d58-ae7f-f80a6f7cfdcd None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:13.612711 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 199/392] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Aug 7 19:13:13 2022] GET /identity/v3/groups/654f30bbad9845df99d839be8f392c4e => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:13.621607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44316b-784d-4c04-9f13-1143aabbdc91 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:13.669583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:13.774131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44316b-784d-4c04-9f13-1143aabbdc91 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:13.826776 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a650f-358a-410a-b56d-a4f6fc2f45b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:13.827345 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a650f-358a-410a-b56d-a4f6fc2f45b5 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:13.827620 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a650f-358a-410a-b56d-a4f6fc2f45b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:13.827893 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a650f-358a-410a-b56d-a4f6fc2f45b5 None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:13.829095 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a650f-358a-410a-b56d-a4f6fc2f45b5 None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:13.830541 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a650f-358a-410a-b56d-a4f6fc2f45b5 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:13.832301 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 194/393] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:13:13 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:13.838679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db1acb-751e-4d58-ae7f-f80a6f7cfdcd None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:13.919818 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4608ab3-655b-43b4-b9c1-c5276644a66d None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:13.920175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4608ab3-655b-43b4-b9c1-c5276644a66d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:13.920411 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4608ab3-655b-43b4-b9c1-c5276644a66d None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:13.922044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 195/394] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:13:13.927382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c66c7-9971-47df-a4e8-4241c97dd44f None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:13.927665 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c66c7-9971-47df-a4e8-4241c97dd44f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:13.927945 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c66c7-9971-47df-a4e8-4241c97dd44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:13.950865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-6e3c66c7-9971-47df-a4e8-4241c97dd44f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:13.952136 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6e3c66c7-9971-47df-a4e8-4241c97dd44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:14.000467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db1acb-751e-4d58-ae7f-f80a6f7cfdcd None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:14.066018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 196/395] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:13:14.088313 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c66c7-9971-47df-a4e8-4241c97dd44f None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:14.092312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1789d7-a6f5-4484-a842-6552bc534cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:14.093057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1789d7-a6f5-4484-a842-6552bc534cc3 None admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:14.093420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1789d7-a6f5-4484-a842-6552bc534cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:14.093745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1789d7-a6f5-4484-a842-6552bc534cc3 None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0/groups/654f30bbad9845df99d839be8f392c4e/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:14.095842 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1789d7-a6f5-4484-a842-6552bc534cc3 None admin] RBAC: Authorizing `identity:create_grant(project_id=cabd760392b6495e87b66b85b6a15ca0, group_id=654f30bbad9845df99d839be8f392c4e, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:14.097503 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1789d7-a6f5-4484-a842-6552bc534cc3 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:14.121137 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:14.130108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 200/396] 10.222.0.17 () {58 vars in 1442 bytes} [Sun Aug 7 19:13:13 2022] PUT /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0/groups/654f30bbad9845df99d839be8f392c4e/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:13:14.192590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615017b7-f648-4d05-a0fb-0a71c6a843e4 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:14.192925 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615017b7-f648-4d05-a0fb-0a71c6a843e4 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:14.193149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615017b7-f648-4d05-a0fb-0a71c6a843e4 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:14.194962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 201/397] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:14.202518 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c66c7-9971-47df-a4e8-4241c97dd44f None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:14.204862 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc6638-98d0-4a50-bc82-62e9e307289f None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:14.205209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc6638-98d0-4a50-bc82-62e9e307289f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:14.205460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc6638-98d0-4a50-bc82-62e9e307289f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:14.224745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:14.241712 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-a9fc6638-98d0-4a50-bc82-62e9e307289f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:14.243499 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a9fc6638-98d0-4a50-bc82-62e9e307289f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:14.247342 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc918a-9826-4810-ae7e-7ab35d98862c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:14.248028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc918a-9826-4810-ae7e-7ab35d98862c None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:14.248311 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc918a-9826-4810-ae7e-7ab35d98862c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:14.248584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc918a-9826-4810-ae7e-7ab35d98862c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:14.272683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc918a-9826-4810-ae7e-7ab35d98862c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:14.278082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-dffc918a-9826-4810-ae7e-7ab35d98862c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:13:14.278082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:14.278082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:14.278082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:14.278082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:14.278082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:14.278082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:14.278082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:14.278082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:14.278082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:14.278082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:14.278082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:14.278082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:14.278082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:14.278082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:14.278082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:14.279030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:14.279030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:14.279030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:14.279030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:14.279030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:14.279030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:14.279030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:14.279030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:14.279030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:14.279030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:14.279030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:14.279030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:14.279030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:14.279030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:14.279030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:14.279030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:14.279816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:14.279816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:14.279816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:14.279816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:14.279816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:14.279816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:14.279816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:14.279816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:14.279816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:14.279816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:14.279816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:13:14.279816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:14.289005 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 197/398] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:14.294456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc918a-9826-4810-ae7e-7ab35d98862c None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:14.398974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 202/399] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:14.403727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:14.415547 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc918a-9826-4810-ae7e-7ab35d98862c None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:14.422924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc6638-98d0-4a50-bc82-62e9e307289f None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:14.460673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7523457-0c83-4e16-927c-ede6f73c9e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:14.461319 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7523457-0c83-4e16-927c-ede6f73c9e92 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:14.461601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7523457-0c83-4e16-927c-ede6f73c9e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:14.461877 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7523457-0c83-4e16-927c-ede6f73c9e92 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:14.462689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7523457-0c83-4e16-927c-ede6f73c9e92 None admin] RBAC: Adding query filter params (name=member) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:14.462996 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7523457-0c83-4e16-927c-ede6f73c9e92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:14.464474 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7523457-0c83-4e16-927c-ede6f73c9e92 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:14.483139 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 198/400] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:13:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:14.491033 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7523457-0c83-4e16-927c-ede6f73c9e92 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:14.594501 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc6638-98d0-4a50-bc82-62e9e307289f None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:14.668367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f5439-27e2-41fe-8519-b57c84ed4787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:14.669768 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f5439-27e2-41fe-8519-b57c84ed4787 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:14.670048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f5439-27e2-41fe-8519-b57c84ed4787 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:14.670322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f5439-27e2-41fe-8519-b57c84ed4787 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:14.674999 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:14.686485 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7523457-0c83-4e16-927c-ede6f73c9e92 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:14.696883 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f5439-27e2-41fe-8519-b57c84ed4787 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:14.702196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-8b5f5439-27e2-41fe-8519-b57c84ed4787 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:13:14.702196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:14.702196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:14.702196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:14.702196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:14.702196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:14.702196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:14.702196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:14.702196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:14.702196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:14.702196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:14.702196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:14.702196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:14.702196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:14.702196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:14.702196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:14.703337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:14.703337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:14.703337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:14.703337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:14.703337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:14.703337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:14.703337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:14.703337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:14.703337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:14.703337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:14.703337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:14.703337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:14.703337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:14.703337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:14.703337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:14.703337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:14.704350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:14.704350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:14.704350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:14.704350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:14.704350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:14.704350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:14.704350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:14.704350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:14.704350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:14.704350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:14.704350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:13:14.704350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:14.716641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 203/401] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:13:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:14.719746 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f5439-27e2-41fe-8519-b57c84ed4787 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:14.750167 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74552a18-e3c5-4c4a-b15c-4875c19ea22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:14.750845 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74552a18-e3c5-4c4a-b15c-4875c19ea22e None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:14.751127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74552a18-e3c5-4c4a-b15c-4875c19ea22e None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:14.751403 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74552a18-e3c5-4c4a-b15c-4875c19ea22e None admin] PATH_INFO: `/v3/groups/654f30bbad9845df99d839be8f392c4e` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:14.754987 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74552a18-e3c5-4c4a-b15c-4875c19ea22e None admin] RBAC: Authorizing `identity:get_group(group_id=654f30bbad9845df99d839be8f392c4e)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:14.756545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74552a18-e3c5-4c4a-b15c-4875c19ea22e None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:14.758767 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 199/402] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Aug 7 19:13:14 2022] GET /identity/v3/groups/654f30bbad9845df99d839be8f392c4e => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:14.768164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74552a18-e3c5-4c4a-b15c-4875c19ea22e None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:14.834476 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:14.875475 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f5439-27e2-41fe-8519-b57c84ed4787 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:14.940780 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5a6c4-0c76-41f5-b8a6-aed4d6fa44e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:14.941318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5a6c4-0c76-41f5-b8a6-aed4d6fa44e5 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:14.941740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5a6c4-0c76-41f5-b8a6-aed4d6fa44e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:14.942018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5a6c4-0c76-41f5-b8a6-aed4d6fa44e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:14.942697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5a6c4-0c76-41f5-b8a6-aed4d6fa44e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:14.942998 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5a6c4-0c76-41f5-b8a6-aed4d6fa44e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:14.944426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5a6c4-0c76-41f5-b8a6-aed4d6fa44e5 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:14.961580 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 204/403] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:14.967617 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5a6c4-0c76-41f5-b8a6-aed4d6fa44e5 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:14.976031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74552a18-e3c5-4c4a-b15c-4875c19ea22e None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:15.044124 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c3bd0-25aa-4f59-b7cf-16a9552d6317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:15.045558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c3bd0-25aa-4f59-b7cf-16a9552d6317 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:15.045859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c3bd0-25aa-4f59-b7cf-16a9552d6317 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:15.046131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c3bd0-25aa-4f59-b7cf-16a9552d6317 None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:15.049065 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c3bd0-25aa-4f59-b7cf-16a9552d6317 None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:15.052497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c3bd0-25aa-4f59-b7cf-16a9552d6317 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:15.054285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 200/404] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:13:14 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:15.061114 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c3bd0-25aa-4f59-b7cf-16a9552d6317 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:15.081545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:15.135093 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5a6c4-0c76-41f5-b8a6-aed4d6fa44e5 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:15.177907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b9f7c-be0c-42f7-b6ee-ed07d296d9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:15.178577 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b9f7c-be0c-42f7-b6ee-ed07d296d9ce None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:15.178855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b9f7c-be0c-42f7-b6ee-ed07d296d9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:15.179184 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b9f7c-be0c-42f7-b6ee-ed07d296d9ce None admin] PATH_INFO: `/v3/groups/654f30bbad9845df99d839be8f392c4e` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:15.180457 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b9f7c-be0c-42f7-b6ee-ed07d296d9ce None admin] RBAC: Authorizing `identity:get_group(group_id=654f30bbad9845df99d839be8f392c4e)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:15.182211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b9f7c-be0c-42f7-b6ee-ed07d296d9ce None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:15.185172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 205/405] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Aug 7 19:13:14 2022] GET /identity/v3/groups/654f30bbad9845df99d839be8f392c4e => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:15.190549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b9f7c-be0c-42f7-b6ee-ed07d296d9ce None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:15.300323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b9f7c-be0c-42f7-b6ee-ed07d296d9ce None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:15.329123 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:15.339681 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54ec7d-11df-4b19-ae21-273e39649217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:15.340664 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54ec7d-11df-4b19-ae21-273e39649217 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:15.341014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54ec7d-11df-4b19-ae21-273e39649217 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:15.341516 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54ec7d-11df-4b19-ae21-273e39649217 None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:15.342759 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a54ec7d-11df-4b19-ae21-273e39649217 None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:15.344452 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a54ec7d-11df-4b19-ae21-273e39649217 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:15.346556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 206/406] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:13:15 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:15.352303 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54ec7d-11df-4b19-ae21-273e39649217 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:15.352449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:15.371536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c3bd0-25aa-4f59-b7cf-16a9552d6317 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:15.430776 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff8240-4891-450d-9421-473c50862f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:15.431542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff8240-4891-450d-9421-473c50862f59 None admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:15.431806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff8240-4891-450d-9421-473c50862f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:15.432153 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff8240-4891-450d-9421-473c50862f59 None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469/groups/654f30bbad9845df99d839be8f392c4e/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:15.434545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff8240-4891-450d-9421-473c50862f59 None admin] RBAC: Authorizing `identity:create_grant(project_id=ad772dbadd1e498f9d58bf65502c4469, group_id=654f30bbad9845df99d839be8f392c4e, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:15.436037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff8240-4891-450d-9421-473c50862f59 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:15.457281 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 201/407] 10.222.0.17 () {58 vars in 1442 bytes} [Sun Aug 7 19:13:15 2022] PUT /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469/groups/654f30bbad9845df99d839be8f392c4e/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:13:15.459200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55a604-3ca7-402c-9c73-b746e81c6a0c None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:15.459746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55a604-3ca7-402c-9c73-b746e81c6a0c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:15.460335 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55a604-3ca7-402c-9c73-b746e81c6a0c None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:15.463138 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 202/408] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:15.477856 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a136fa2-0f13-4fa2-b572-0ee8535f641c None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:15.478533 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a136fa2-0f13-4fa2-b572-0ee8535f641c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:15.479029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a136fa2-0f13-4fa2-b572-0ee8535f641c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:15.510638 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-6a136fa2-0f13-4fa2-b572-0ee8535f641c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:15.513647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6a136fa2-0f13-4fa2-b572-0ee8535f641c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:15.544039 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54ec7d-11df-4b19-ae21-273e39649217 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:15.604086 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4f6dd-482a-4303-b0bd-d8a98b9a349e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:15.605149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4f6dd-482a-4303-b0bd-d8a98b9a349e None admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:15.605703 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4f6dd-482a-4303-b0bd-d8a98b9a349e None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:15.606180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4f6dd-482a-4303-b0bd-d8a98b9a349e None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469/groups/654f30bbad9845df99d839be8f392c4e/roles/9a1c954216e34607a6788f44f07a593b` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:15.608140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 203/409] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:13:15.609826 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3ff92-d8a8-4dcb-8003-12d26bcf6df3 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:15.610357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3ff92-d8a8-4dcb-8003-12d26bcf6df3 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:15.610823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3ff92-d8a8-4dcb-8003-12d26bcf6df3 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:15.613316 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 204/410] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:15.615018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4f6dd-482a-4303-b0bd-d8a98b9a349e None admin] RBAC: Authorizing `identity:create_grant(project_id=ad772dbadd1e498f9d58bf65502c4469, group_id=654f30bbad9845df99d839be8f392c4e, role_id=9a1c954216e34607a6788f44f07a593b)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:15.616880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4f6dd-482a-4303-b0bd-d8a98b9a349e None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:15.626537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4213b7-74d4-4298-b62a-bb89f2ba7a8a None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:15.627172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4213b7-74d4-4298-b62a-bb89f2ba7a8a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:15.627640 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4213b7-74d4-4298-b62a-bb89f2ba7a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:15.656024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-9a4213b7-74d4-4298-b62a-bb89f2ba7a8a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:15.656024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9a4213b7-74d4-4298-b62a-bb89f2ba7a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:15.656583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 207/411] 10.222.0.17 () {58 vars in 1442 bytes} [Sun Aug 7 19:13:15 2022] PUT /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469/groups/654f30bbad9845df99d839be8f392c4e/roles/9a1c954216e34607a6788f44f07a593b => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:13:15.657784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4f6dd-482a-4303-b0bd-d8a98b9a349e None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:15.751305 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:15.751566 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 205/412] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:15.804879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4213b7-74d4-4298-b62a-bb89f2ba7a8a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:15.821636 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4f6dd-482a-4303-b0bd-d8a98b9a349e None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:15.836007 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:15.869187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079bb52-0d9c-4fe0-9037-e58e3e50ef0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:15.869753 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079bb52-0d9c-4fe0-9037-e58e3e50ef0e None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:15.870039 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079bb52-0d9c-4fe0-9037-e58e3e50ef0e None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:15.870291 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079bb52-0d9c-4fe0-9037-e58e3e50ef0e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:15.885696 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079bb52-0d9c-4fe0-9037-e58e3e50ef0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:15.892583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-c079bb52-0d9c-4fe0-9037-e58e3e50ef0e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:13:15.892583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:15.892583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:15.892583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:15.892583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:15.892583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:15.892583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:15.892583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:15.892583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:15.892583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:15.892583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:15.892583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:15.892583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:15.892583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:15.892583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:15.892583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:15.893548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:15.893548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:15.893548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:15.893548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:15.893548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:15.893548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:15.893548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:15.893548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:15.893548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:15.893548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:15.893548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:15.893548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:15.893548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:15.893548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:15.893548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:15.893548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:15.894341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:15.894341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:15.894341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:15.894341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:15.894341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:15.894341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:15.894341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:15.894341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:15.894341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:15.894341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:15.894341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:13:15.894341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:15.894949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 208/413] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:13:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:15.901992 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079bb52-0d9c-4fe0-9037-e58e3e50ef0e None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:15.959381 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4213b7-74d4-4298-b62a-bb89f2ba7a8a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:16.007536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd572bc8-576b-4291-98eb-4a781e216667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:16.009803 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd572bc8-576b-4291-98eb-4a781e216667 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:16.010100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd572bc8-576b-4291-98eb-4a781e216667 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:16.010377 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd572bc8-576b-4291-98eb-4a781e216667 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:16.042152 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079bb52-0d9c-4fe0-9037-e58e3e50ef0e None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:16.044654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd572bc8-576b-4291-98eb-4a781e216667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:16.049531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-bd572bc8-576b-4291-98eb-4a781e216667 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:13:16.049531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:16.049531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:16.049531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:16.049531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:16.049531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:16.049531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:16.049531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:16.049531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:16.049531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:16.049531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:16.049531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:16.049531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:16.049531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:16.049531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:16.049531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:16.050432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:16.050432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:16.050432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:16.050432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:16.050432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:16.050432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:16.050432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:16.050432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:16.050432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:16.050432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:16.050432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:16.050432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:16.050432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:16.050432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:16.050432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:16.050432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:16.051229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:16.051229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:16.051229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:16.051229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:16.051229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:16.051229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:16.051229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:16.051229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:16.051229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:16.051229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:16.051229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:13:16.051229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:16.051229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 206/414] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:13:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:16.061077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd572bc8-576b-4291-98eb-4a781e216667 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:16.069758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:16.098128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2f648-7ea6-426d-9ed3-df74d8515a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:16.098793 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2f648-7ea6-426d-9ed3-df74d8515a96 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:16.099057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2f648-7ea6-426d-9ed3-df74d8515a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:16.099328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2f648-7ea6-426d-9ed3-df74d8515a96 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:16.100027 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2f648-7ea6-426d-9ed3-df74d8515a96 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:16.100321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2f648-7ea6-426d-9ed3-df74d8515a96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:16.101726 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2f648-7ea6-426d-9ed3-df74d8515a96 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:16.118087 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 209/415] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:16.123449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2f648-7ea6-426d-9ed3-df74d8515a96 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:16.178533 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:16.230696 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd572bc8-576b-4291-98eb-4a781e216667 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:16.259402 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2f648-7ea6-426d-9ed3-df74d8515a96 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:16.294357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dca57d-b4e5-4da7-8792-e0a79286d6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:16.295433 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dca57d-b4e5-4da7-8792-e0a79286d6be None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:16.295713 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dca57d-b4e5-4da7-8792-e0a79286d6be None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:16.295983 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dca57d-b4e5-4da7-8792-e0a79286d6be None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:16.296684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dca57d-b4e5-4da7-8792-e0a79286d6be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:16.296984 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dca57d-b4e5-4da7-8792-e0a79286d6be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:16.299278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dca57d-b4e5-4da7-8792-e0a79286d6be None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:16.301013 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa98d88-f617-4f7f-869f-a2e27846b5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:16.302372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa98d88-f617-4f7f-869f-a2e27846b5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:16.302862 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa98d88-f617-4f7f-869f-a2e27846b5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:16.303327 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa98d88-f617-4f7f-869f-a2e27846b5b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:16.317684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 207/416] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:16.325918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dca57d-b4e5-4da7-8792-e0a79286d6be None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:16.398746 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:16.428346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa98d88-f617-4f7f-869f-a2e27846b5b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:16.429857 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa98d88-f617-4f7f-869f-a2e27846b5b3 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:16.483868 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dca57d-b4e5-4da7-8792-e0a79286d6be None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:16.515159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:16.522077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc591b-41a7-4c0f-bf7b-08a24c9e4228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:16.523056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc591b-41a7-4c0f-bf7b-08a24c9e4228 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:16.523548 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc591b-41a7-4c0f-bf7b-08a24c9e4228 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:16.524037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc591b-41a7-4c0f-bf7b-08a24c9e4228 None admin] PATH_INFO: `/v3/groups/62f4263724184b8282a082eacc7ac6ac` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:16.525386 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc591b-41a7-4c0f-bf7b-08a24c9e4228 None admin] RBAC: Authorizing `identity:get_group(group_id=62f4263724184b8282a082eacc7ac6ac)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:16.527056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc591b-41a7-4c0f-bf7b-08a24c9e4228 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:16.529359 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 208/417] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Aug 7 19:13:16 2022] GET /identity/v3/groups/62f4263724184b8282a082eacc7ac6ac => generated 229 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:16.534463 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc591b-41a7-4c0f-bf7b-08a24c9e4228 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:16.542845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 210/418] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 19:13:16.549078 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa98d88-f617-4f7f-869f-a2e27846b5b3 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:16.636125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:16.664262 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa98d88-f617-4f7f-869f-a2e27846b5b3 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:16.684395 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc591b-41a7-4c0f-bf7b-08a24c9e4228 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:16.703501 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b296a7-1b66-446f-842d-5741c7e40adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:16.704177 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b296a7-1b66-446f-842d-5741c7e40adc None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:16.704482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b296a7-1b66-446f-842d-5741c7e40adc None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:16.704997 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b296a7-1b66-446f-842d-5741c7e40adc None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:16.705880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b296a7-1b66-446f-842d-5741c7e40adc None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:16.707346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b296a7-1b66-446f-842d-5741c7e40adc None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:16.713805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 211/419] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:13:16 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:16.715393 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d7715-c1db-4350-99bc-2580b5c47f45 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:16.715631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d7715-c1db-4350-99bc-2580b5c47f45 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:16.715909 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d7715-c1db-4350-99bc-2580b5c47f45 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:16.717799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 212/420] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:16.722087 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d7715-c1db-4350-99bc-2580b5c47f45 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:16.735065 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97557df9-5f7b-4fad-becd-9686f1fab63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:16.736504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97557df9-5f7b-4fad-becd-9686f1fab63a None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:16.737244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97557df9-5f7b-4fad-becd-9686f1fab63a None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:16.737744 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97557df9-5f7b-4fad-becd-9686f1fab63a None admin] PATH_INFO: `/v3/projects/6c00a4064cc8477492c9b8b2e5412197` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:16.739559 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97557df9-5f7b-4fad-becd-9686f1fab63a None admin] RBAC: Authorizing `identity:get_project(project_id=6c00a4064cc8477492c9b8b2e5412197)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:16.741619 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97557df9-5f7b-4fad-becd-9686f1fab63a None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:16.744887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 209/421] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:13:16 2022] GET /identity/v3/projects/6c00a4064cc8477492c9b8b2e5412197 => generated 344 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:16.747059 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fde9f-d41e-42ff-acc9-b6d02169d094 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:16.747575 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fde9f-d41e-42ff-acc9-b6d02169d094 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:16.748227 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fde9f-d41e-42ff-acc9-b6d02169d094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:16.772750 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-2d8fde9f-d41e-42ff-acc9-b6d02169d094 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:16.774608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-2d8fde9f-d41e-42ff-acc9-b6d02169d094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:16.835522 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d7715-c1db-4350-99bc-2580b5c47f45 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:16.863422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:16.876016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 210/422] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:16.876016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8fde9f-d41e-42ff-acc9-b6d02169d094 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:16.892557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f12a1-92e7-4ef1-bd8a-db5f5c5df641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:16.893719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f12a1-92e7-4ef1-bd8a-db5f5c5df641 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:16.894270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f12a1-92e7-4ef1-bd8a-db5f5c5df641 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:16.894761 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f12a1-92e7-4ef1-bd8a-db5f5c5df641 None admin] PATH_INFO: `/v3/groups/654f30bbad9845df99d839be8f392c4e` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:16.898772 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f12a1-92e7-4ef1-bd8a-db5f5c5df641 None admin] RBAC: Authorizing `identity:get_group(group_id=654f30bbad9845df99d839be8f392c4e)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:16.900617 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f12a1-92e7-4ef1-bd8a-db5f5c5df641 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:16.903163 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 213/423] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Aug 7 19:13:16 2022] GET /identity/v3/groups/654f30bbad9845df99d839be8f392c4e => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:16.913257 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f12a1-92e7-4ef1-bd8a-db5f5c5df641 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:16.928183 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:16.998079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8fde9f-d41e-42ff-acc9-b6d02169d094 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:17.026278 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f12a1-92e7-4ef1-bd8a-db5f5c5df641 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:17.045337 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba31de7-0662-46ba-a111-d22cd400ab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:17.046044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba31de7-0662-46ba-a111-d22cd400ab95 None admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:17.046321 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba31de7-0662-46ba-a111-d22cd400ab95 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:17.046596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba31de7-0662-46ba-a111-d22cd400ab95 None admin] PATH_INFO: `/v3/projects/6c00a4064cc8477492c9b8b2e5412197/groups/62f4263724184b8282a082eacc7ac6ac/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:17.048682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba31de7-0662-46ba-a111-d22cd400ab95 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c00a4064cc8477492c9b8b2e5412197, group_id=62f4263724184b8282a082eacc7ac6ac, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:17.050145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba31de7-0662-46ba-a111-d22cd400ab95 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:17.068061 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 211/424] 10.222.0.17 () {58 vars in 1442 bytes} [Sun Aug 7 19:13:16 2022] PUT /identity/v3/projects/6c00a4064cc8477492c9b8b2e5412197/groups/62f4263724184b8282a082eacc7ac6ac/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:13:17.068998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba31de7-0662-46ba-a111-d22cd400ab95 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:17.079040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a7b0a-dc2b-4514-809c-3c0fcd6f0e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:17.079707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a7b0a-dc2b-4514-809c-3c0fcd6f0e19 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:17.079981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a7b0a-dc2b-4514-809c-3c0fcd6f0e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:17.080369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a7b0a-dc2b-4514-809c-3c0fcd6f0e19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:17.104712 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3a7b0a-dc2b-4514-809c-3c0fcd6f0e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:17.111146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-dd3a7b0a-dc2b-4514-809c-3c0fcd6f0e19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:13:17.111146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:17.111146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:17.111146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:17.111146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:17.111146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:17.111146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:17.111146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:17.111146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:17.111146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:17.111146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:17.111146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:17.111146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:17.111146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:17.111146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:17.111146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:17.112218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:17.112218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:17.112218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:17.112218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:17.112218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:17.112218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:17.112218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:17.112218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:17.112218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:17.112218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:17.112218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:17.112218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:17.112218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:17.112218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:17.112218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:17.112218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:17.113011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:17.113011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:17.113011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:17.113011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:17.113011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:17.113011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:17.113011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:17.113011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:17.113011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:17.113011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:17.113011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:13:17.113011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:17.120953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 214/425] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:17.126889 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a7b0a-dc2b-4514-809c-3c0fcd6f0e19 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:17.142183 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:17.186699 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba31de7-0662-46ba-a111-d22cd400ab95 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:17.226670 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c2eca-01bd-40ef-ad85-03d57f8b1704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:17.227469 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c2eca-01bd-40ef-ad85-03d57f8b1704 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:17.227750 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c2eca-01bd-40ef-ad85-03d57f8b1704 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:17.228055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c2eca-01bd-40ef-ad85-03d57f8b1704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:17.228813 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c2eca-01bd-40ef-ad85-03d57f8b1704 None admin] RBAC: Adding query filter params (group.id=654f30bbad9845df99d839be8f392c4e, role.id=9a1c954216e34607a6788f44f07a593b, scope.project.id=cabd760392b6495e87b66b85b6a15ca0) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:17.230122 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c2eca-01bd-40ef-ad85-03d57f8b1704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:17.231736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c2eca-01bd-40ef-ad85-03d57f8b1704 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:17.244801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 212/426] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Aug 7 19:13:17 2022] GET /identity/v3/role_assignments?group.id=654f30bbad9845df99d839be8f392c4e&scope.project.id=cabd760392b6495e87b66b85b6a15ca0&role.id=9a1c954216e34607a6788f44f07a593b => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:17.317949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:17.359672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a7b0a-dc2b-4514-809c-3c0fcd6f0e19 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:17.400592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45b3a5-110f-449a-8cb8-551362f28912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:17.401243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45b3a5-110f-449a-8cb8-551362f28912 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:17.401520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45b3a5-110f-449a-8cb8-551362f28912 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:17.401792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45b3a5-110f-449a-8cb8-551362f28912 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:17.402570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee45b3a5-110f-449a-8cb8-551362f28912 None admin] RBAC: Adding query filter params (name=member) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:17.402876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee45b3a5-110f-449a-8cb8-551362f28912 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:17.404373 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee45b3a5-110f-449a-8cb8-551362f28912 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:17.414914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 215/427] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:13:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:17.420453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c2eca-01bd-40ef-ad85-03d57f8b1704 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:17.534602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c2eca-01bd-40ef-ad85-03d57f8b1704 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:17.563388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:17.576146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561c536-3a11-4d26-98d6-97ff3bfb5921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:17.576784 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561c536-3a11-4d26-98d6-97ff3bfb5921 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:17.577068 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561c536-3a11-4d26-98d6-97ff3bfb5921 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:17.577342 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561c536-3a11-4d26-98d6-97ff3bfb5921 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:17.657009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561c536-3a11-4d26-98d6-97ff3bfb5921 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:17.658484 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561c536-3a11-4d26-98d6-97ff3bfb5921 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:17.769959 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 213/428] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:17.777138 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45b3a5-110f-449a-8cb8-551362f28912 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:17.833816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb92629-cb33-40c5-b3f0-45c14fcc5748 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:17.834116 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb92629-cb33-40c5-b3f0-45c14fcc5748 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:17.834270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb92629-cb33-40c5-b3f0-45c14fcc5748 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:17.835793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 214/429] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:17.842049 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75769e2-c422-49cd-89ec-f775db57ec0f None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:17.842049 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75769e2-c422-49cd-89ec-f775db57ec0f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:17.842049 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75769e2-c422-49cd-89ec-f775db57ec0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:17.861890 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-a75769e2-c422-49cd-89ec-f775db57ec0f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:17.863133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-a75769e2-c422-49cd-89ec-f775db57ec0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:17.892600 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45b3a5-110f-449a-8cb8-551362f28912 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:17.900392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:17.923932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:17.930017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3817dd-c4ea-49ca-8d53-69f652b42767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:17.931093 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3817dd-c4ea-49ca-8d53-69f652b42767 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:17.931714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3817dd-c4ea-49ca-8d53-69f652b42767 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:17.932265 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3817dd-c4ea-49ca-8d53-69f652b42767 None admin] PATH_INFO: `/v3/projects/cabd760392b6495e87b66b85b6a15ca0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:17.933980 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3817dd-c4ea-49ca-8d53-69f652b42767 None admin] RBAC: Authorizing `identity:get_project(project_id=cabd760392b6495e87b66b85b6a15ca0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:17.935643 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3817dd-c4ea-49ca-8d53-69f652b42767 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:17.937952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 216/430] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:13:17 2022] GET /identity/v3/projects/cabd760392b6495e87b66b85b6a15ca0 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:17.943128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3817dd-c4ea-49ca-8d53-69f652b42767 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:17.943884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 215/431] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:13:17.967500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75769e2-c422-49cd-89ec-f775db57ec0f None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:18.083316 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75769e2-c422-49cd-89ec-f775db57ec0f None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:18.083905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3817dd-c4ea-49ca-8d53-69f652b42767 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:18.122589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30cd482-35de-43b6-ba3e-7a656fcc9224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:18.122969 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347330f-1446-420f-ae46-667287d92493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:18.123424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cd482-35de-43b6-ba3e-7a656fcc9224 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:18.123503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cd482-35de-43b6-ba3e-7a656fcc9224 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:18.123588 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347330f-1446-420f-ae46-667287d92493 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:18.123766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30cd482-35de-43b6-ba3e-7a656fcc9224 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:18.123855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347330f-1446-420f-ae46-667287d92493 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:18.124120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347330f-1446-420f-ae46-667287d92493 None admin] PATH_INFO: `/v3/groups/654f30bbad9845df99d839be8f392c4e` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:18.125236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347330f-1446-420f-ae46-667287d92493 None admin] RBAC: Authorizing `identity:get_group(group_id=654f30bbad9845df99d839be8f392c4e)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:18.126806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347330f-1446-420f-ae46-667287d92493 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:18.133335 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 217/432] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Aug 7 19:13:17 2022] GET /identity/v3/groups/654f30bbad9845df99d839be8f392c4e => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:18.134668 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1e3de-98c0-48e6-b3c1-4d71475c14be None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:18.134954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1e3de-98c0-48e6-b3c1-4d71475c14be None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:18.135104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30cd482-35de-43b6-ba3e-7a656fcc9224 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:18.135216 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1e3de-98c0-48e6-b3c1-4d71475c14be None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:18.140135 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 218/433] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:18.140135 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1e3de-98c0-48e6-b3c1-4d71475c14be None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:18.140334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-f30cd482-35de-43b6-ba3e-7a656fcc9224 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:13:18.140334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:18.140334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:18.140334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:18.140334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:18.140334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:18.140334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:18.140334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:18.140334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:18.140334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:18.140334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:18.140334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:18.140334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:18.140334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:18.140334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:18.140334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:18.141205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:18.141205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:18.141205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:18.141205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:18.141205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:18.141205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:18.141205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:18.141205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:18.141205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:18.141205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:18.141205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:18.141205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:18.141205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:18.141205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:18.141205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:18.141205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:18.142146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:18.142146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:18.142146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:18.142146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:18.142146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:18.142146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:18.142146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:18.142146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:18.142146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:18.142146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:18.142146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 19:13:18.142146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:18.146082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 216/434] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:18.147451 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71812a-bbf6-4e1e-a63f-0c23973df2ad None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:18.147734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71812a-bbf6-4e1e-a63f-0c23973df2ad None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:18.148035 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71812a-bbf6-4e1e-a63f-0c23973df2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:18.166315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-fc71812a-bbf6-4e1e-a63f-0c23973df2ad None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:18.167564 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-fc71812a-bbf6-4e1e-a63f-0c23973df2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:18.175595 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:18.222902 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:18.241591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 217/435] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:18.242473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71812a-bbf6-4e1e-a63f-0c23973df2ad None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:18.271054 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1e3de-98c0-48e6-b3c1-4d71475c14be None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:18.307056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43712b-4ed3-4e44-8cba-451e36cda2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:18.308074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43712b-4ed3-4e44-8cba-451e36cda2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:18.308355 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43712b-4ed3-4e44-8cba-451e36cda2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:18.308690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b43712b-4ed3-4e44-8cba-451e36cda2d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:18.309423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43712b-4ed3-4e44-8cba-451e36cda2d2 None admin] RBAC: Adding query filter params (group.id=654f30bbad9845df99d839be8f392c4e, role.id=246bbd6878334bba893bc6f5bf5caee0, scope.project.id=cabd760392b6495e87b66b85b6a15ca0) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:18.309759 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43712b-4ed3-4e44-8cba-451e36cda2d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:18.311243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b43712b-4ed3-4e44-8cba-451e36cda2d2 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:18.318231 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 219/436] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Aug 7 19:13:18 2022] GET /identity/v3/role_assignments?group.id=654f30bbad9845df99d839be8f392c4e&scope.project.id=cabd760392b6495e87b66b85b6a15ca0&role.id=246bbd6878334bba893bc6f5bf5caee0 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:18.318848 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43712b-4ed3-4e44-8cba-451e36cda2d2 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:18.374635 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71812a-bbf6-4e1e-a63f-0c23973df2ad None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:18.429384 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5ee7f-f6d7-404d-abdd-5e2def8518bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:18.429944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5ee7f-f6d7-404d-abdd-5e2def8518bf None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:18.430230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5ee7f-f6d7-404d-abdd-5e2def8518bf None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:18.430506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5ee7f-f6d7-404d-abdd-5e2def8518bf None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:18.431199 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5ee7f-f6d7-404d-abdd-5e2def8518bf None admin] RBAC: Adding query filter params (name=member) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:18.431513 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5ee7f-f6d7-404d-abdd-5e2def8518bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:18.433425 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5ee7f-f6d7-404d-abdd-5e2def8518bf None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:18.443519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 218/437] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:13:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:18.448994 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5ee7f-f6d7-404d-abdd-5e2def8518bf None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:18.454006 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:18.461699 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b43712b-4ed3-4e44-8cba-451e36cda2d2 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:18.510968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:18.518154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbfd46-bd91-4ff4-ba2a-90ff46e81088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:18.518810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbfd46-bd91-4ff4-ba2a-90ff46e81088 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:18.519085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbfd46-bd91-4ff4-ba2a-90ff46e81088 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:18.519354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbfd46-bd91-4ff4-ba2a-90ff46e81088 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:18.541446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbfd46-bd91-4ff4-ba2a-90ff46e81088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:18.548221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-80cbfd46-bd91-4ff4-ba2a-90ff46e81088 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:13:18.548221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:18.548221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:18.548221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:18.548221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:18.548221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:18.548221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:18.548221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:18.548221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:18.548221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:18.548221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:18.548221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:18.548221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:18.548221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:18.548221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:18.548221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:18.549153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:18.549153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:18.549153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:18.549153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:18.549153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:18.549153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:18.549153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:18.549153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:18.549153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:18.549153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:18.549153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:18.549153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:18.549153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:18.549153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:18.549153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:18.549153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:18.550023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:18.550023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:18.550023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:18.550023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:18.550023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:18.550023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:18.550023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:18.550023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:18.550023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:18.550023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:18.550023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:13:18.550023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:18.554403 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 220/438] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:13:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:18.555737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbfd46-bd91-4ff4-ba2a-90ff46e81088 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:18.602252 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5ee7f-f6d7-404d-abdd-5e2def8518bf None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:18.646362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442ccb5-1805-442d-8a11-9c0171a9c635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:18.647008 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442ccb5-1805-442d-8a11-9c0171a9c635 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:18.647294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442ccb5-1805-442d-8a11-9c0171a9c635 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:18.647565 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442ccb5-1805-442d-8a11-9c0171a9c635 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:18.700255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbfd46-bd91-4ff4-ba2a-90ff46e81088 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:18.746799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae882fb-1f8f-4939-b343-d015d83ce949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:18.747431 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae882fb-1f8f-4939-b343-d015d83ce949 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:18.747711 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae882fb-1f8f-4939-b343-d015d83ce949 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:18.747981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae882fb-1f8f-4939-b343-d015d83ce949 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:18.748688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae882fb-1f8f-4939-b343-d015d83ce949 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:18.748990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae882fb-1f8f-4939-b343-d015d83ce949 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:18.750397 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae882fb-1f8f-4939-b343-d015d83ce949 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:18.750859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5442ccb5-1805-442d-8a11-9c0171a9c635 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:18.752506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5442ccb5-1805-442d-8a11-9c0171a9c635 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:18.761911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 221/439] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:18.763061 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:18.768424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae882fb-1f8f-4939-b343-d015d83ce949 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:18.845135 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:18.878575 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 219/440] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:18.879801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442ccb5-1805-442d-8a11-9c0171a9c635 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:18.911071 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae882fb-1f8f-4939-b343-d015d83ce949 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:18.959078 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247756f-1b71-4814-afa6-3dfcb0e93ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:18.960294 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247756f-1b71-4814-afa6-3dfcb0e93ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:18.961093 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247756f-1b71-4814-afa6-3dfcb0e93ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:18.961847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247756f-1b71-4814-afa6-3dfcb0e93ce3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:19.011860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442ccb5-1805-442d-8a11-9c0171a9c635 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:19.015626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:19.059819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6911b-dbdf-4ada-9043-cdc95ffecf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:19.060513 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6911b-dbdf-4ada-9043-cdc95ffecf2a None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:19.060798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6911b-dbdf-4ada-9043-cdc95ffecf2a None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:19.061071 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6911b-dbdf-4ada-9043-cdc95ffecf2a None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:19.061376 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247756f-1b71-4814-afa6-3dfcb0e93ce3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:19.062225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6911b-dbdf-4ada-9043-cdc95ffecf2a None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:19.063016 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247756f-1b71-4814-afa6-3dfcb0e93ce3 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:19.063652 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6911b-dbdf-4ada-9043-cdc95ffecf2a None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:19.065715 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 220/441] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:13:18 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:19.071513 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6911b-dbdf-4ada-9043-cdc95ffecf2a None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:19.126149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:19.176462 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 222/442] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:19.182224 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6911b-dbdf-4ada-9043-cdc95ffecf2a None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:19.182410 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247756f-1b71-4814-afa6-3dfcb0e93ce3 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:19.220809 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b931397-fab5-4082-b7fc-34bb12704899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:19.221463 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b931397-fab5-4082-b7fc-34bb12704899 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:19.221743 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b931397-fab5-4082-b7fc-34bb12704899 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:19.222017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b931397-fab5-4082-b7fc-34bb12704899 None admin] PATH_INFO: `/v3/groups/654f30bbad9845df99d839be8f392c4e` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:19.223655 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b931397-fab5-4082-b7fc-34bb12704899 None admin] RBAC: Authorizing `identity:get_group(group_id=654f30bbad9845df99d839be8f392c4e)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:19.225524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b931397-fab5-4082-b7fc-34bb12704899 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:19.227768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 221/443] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Aug 7 19:13:19 2022] GET /identity/v3/groups/654f30bbad9845df99d839be8f392c4e => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:19.233558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b931397-fab5-4082-b7fc-34bb12704899 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:19.277038 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:19.290030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247756f-1b71-4814-afa6-3dfcb0e93ce3 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:19.326855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945999c6-5588-4a06-ae09-8e3ba029eaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:19.327785 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945999c6-5588-4a06-ae09-8e3ba029eaa1 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:19.328596 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945999c6-5588-4a06-ae09-8e3ba029eaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:19.328876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945999c6-5588-4a06-ae09-8e3ba029eaa1 None admin] PATH_INFO: `/v3/projects/ad772dbadd1e498f9d58bf65502c4469` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:19.330466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945999c6-5588-4a06-ae09-8e3ba029eaa1 None admin] RBAC: Authorizing `identity:get_project(project_id=ad772dbadd1e498f9d58bf65502c4469)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:19.331948 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945999c6-5588-4a06-ae09-8e3ba029eaa1 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:19.333629 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 223/444] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:13:19 2022] GET /identity/v3/projects/ad772dbadd1e498f9d58bf65502c4469 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:19.339105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945999c6-5588-4a06-ae09-8e3ba029eaa1 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:19.342535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b931397-fab5-4082-b7fc-34bb12704899 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:19.367047 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:19.379966 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f8bde-f46c-4853-b944-29c579090cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:19.380613 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f8bde-f46c-4853-b944-29c579090cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:19.380910 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f8bde-f46c-4853-b944-29c579090cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:19.381179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f8bde-f46c-4853-b944-29c579090cf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:19.381916 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f8bde-f46c-4853-b944-29c579090cf6 None admin] RBAC: Adding query filter params (group.id=654f30bbad9845df99d839be8f392c4e, role.id=246bbd6878334bba893bc6f5bf5caee0, scope.project.id=ad772dbadd1e498f9d58bf65502c4469) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:19.382228 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f8bde-f46c-4853-b944-29c579090cf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:19.384234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f8bde-f46c-4853-b944-29c579090cf6 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:19.391448 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 222/445] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Aug 7 19:13:19 2022] GET /identity/v3/role_assignments?group.id=654f30bbad9845df99d839be8f392c4e&scope.project.id=ad772dbadd1e498f9d58bf65502c4469&role.id=246bbd6878334bba893bc6f5bf5caee0 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:19.408160 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d5e77-4bcd-4a16-b9dc-11b287bdc63d None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:19.408455 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d5e77-4bcd-4a16-b9dc-11b287bdc63d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:19.408732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d5e77-4bcd-4a16-b9dc-11b287bdc63d None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:19.410208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 223/446] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:19.416432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268173a-3025-4ce9-a76d-856caed1c81a None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:19.416752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268173a-3025-4ce9-a76d-856caed1c81a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:19.417035 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268173a-3025-4ce9-a76d-856caed1c81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:19.439613 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-2268173a-3025-4ce9-a76d-856caed1c81a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:19.440803 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-2268173a-3025-4ce9-a76d-856caed1c81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:19.463434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945999c6-5588-4a06-ae09-8e3ba029eaa1 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:19.507738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83adef-4fe5-489c-8e4b-6263d3c6f3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:19.508434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83adef-4fe5-489c-8e4b-6263d3c6f3ed None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:19.508755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83adef-4fe5-489c-8e4b-6263d3c6f3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:19.508987 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83adef-4fe5-489c-8e4b-6263d3c6f3ed None admin] PATH_INFO: `/v3/groups/654f30bbad9845df99d839be8f392c4e` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:19.510238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83adef-4fe5-489c-8e4b-6263d3c6f3ed None admin] RBAC: Authorizing `identity:get_group(group_id=654f30bbad9845df99d839be8f392c4e)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:19.511822 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83adef-4fe5-489c-8e4b-6263d3c6f3ed None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:19.513533 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 224/447] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Aug 7 19:13:19 2022] GET /identity/v3/groups/654f30bbad9845df99d839be8f392c4e => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:19.519543 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83adef-4fe5-489c-8e4b-6263d3c6f3ed None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:19.532341 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 224/448] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:13:19.544826 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268173a-3025-4ce9-a76d-856caed1c81a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:19.587641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:19.632536 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:19.643896 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83adef-4fe5-489c-8e4b-6263d3c6f3ed None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:19.669099 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268173a-3025-4ce9-a76d-856caed1c81a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:19.684667 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96de48-ba98-405a-9ea2-4d2a72dfdf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:19.685291 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96de48-ba98-405a-9ea2-4d2a72dfdf8a None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:19.685568 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96de48-ba98-405a-9ea2-4d2a72dfdf8a None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:19.686532 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96de48-ba98-405a-9ea2-4d2a72dfdf8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:19.686642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96de48-ba98-405a-9ea2-4d2a72dfdf8a None admin] RBAC: Adding query filter params (group.id=654f30bbad9845df99d839be8f392c4e, role.id=9a1c954216e34607a6788f44f07a593b, scope.project.id=ad772dbadd1e498f9d58bf65502c4469) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:19.686868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96de48-ba98-405a-9ea2-4d2a72dfdf8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:19.689071 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96de48-ba98-405a-9ea2-4d2a72dfdf8a None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:19.695458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 225/449] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Aug 7 19:13:19 2022] GET /identity/v3/role_assignments?group.id=654f30bbad9845df99d839be8f392c4e&scope.project.id=ad772dbadd1e498f9d58bf65502c4469&role.id=9a1c954216e34607a6788f44f07a593b => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:19.711249 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a23b5e-2e44-4429-88ed-806a825b043f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:19.711927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a23b5e-2e44-4429-88ed-806a825b043f None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:19.712246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a23b5e-2e44-4429-88ed-806a825b043f None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:19.712522 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a23b5e-2e44-4429-88ed-806a825b043f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:19.726662 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a23b5e-2e44-4429-88ed-806a825b043f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:19.731849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-56a23b5e-2e44-4429-88ed-806a825b043f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:13:19.731849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:19.731849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:19.731849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:19.731849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:19.731849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:19.731849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:19.731849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:19.731849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:19.731849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:19.731849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:19.731849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:19.731849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:19.731849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:19.731849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:19.731849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:19.732869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:19.732869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:19.732869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:19.732869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:19.732869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:19.732869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:19.732869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:19.732869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:19.732869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:19.732869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:19.732869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:19.732869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:19.732869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:19.732869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:19.732869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:19.732869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:19.733666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:19.733666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:19.733666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:19.733666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:19.733666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:19.733666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:19.733666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:19.733666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:19.733666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:19.733666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:19.733666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:13:19.733666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:19.736081 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 225/450] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:13:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:19.740150 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96de48-ba98-405a-9ea2-4d2a72dfdf8a None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:19.853342 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96de48-ba98-405a-9ea2-4d2a72dfdf8a None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:19.888455 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d391fed-4491-472d-9b48-f4bfae93da31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:19.889087 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d391fed-4491-472d-9b48-f4bfae93da31 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:19.889358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d391fed-4491-472d-9b48-f4bfae93da31 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:19.889621 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d391fed-4491-472d-9b48-f4bfae93da31 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:19.890284 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d391fed-4491-472d-9b48-f4bfae93da31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:19.890573 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d391fed-4491-472d-9b48-f4bfae93da31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:19.891945 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d391fed-4491-472d-9b48-f4bfae93da31 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:19.900469 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 226/451] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:19.906460 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a23b5e-2e44-4429-88ed-806a825b043f None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:20.005419 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:20.012973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a23b5e-2e44-4429-88ed-806a825b043f None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:20.050641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cb729-2cde-4558-b1fc-008f249c879e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:20.051265 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cb729-2cde-4558-b1fc-008f249c879e None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:20.051546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cb729-2cde-4558-b1fc-008f249c879e None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:20.051818 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770cb729-2cde-4558-b1fc-008f249c879e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:20.120867 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770cb729-2cde-4558-b1fc-008f249c879e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:20.122396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770cb729-2cde-4558-b1fc-008f249c879e None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:20.219549 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:20.228453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 226/452] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:20.234645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d391fed-4491-472d-9b48-f4bfae93da31 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:20.256996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:20.342147 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d391fed-4491-472d-9b48-f4bfae93da31 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:20.376506 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc910af9-a9c8-476f-aad7-33eebf09c0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:20.377140 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc910af9-a9c8-476f-aad7-33eebf09c0bb None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:20.377416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc910af9-a9c8-476f-aad7-33eebf09c0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:20.377679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc910af9-a9c8-476f-aad7-33eebf09c0bb None admin] PATH_INFO: `/v3/projects/6c00a4064cc8477492c9b8b2e5412197` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:20.378850 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc910af9-a9c8-476f-aad7-33eebf09c0bb None admin] RBAC: Authorizing `identity:get_project(project_id=6c00a4064cc8477492c9b8b2e5412197)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:20.380322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc910af9-a9c8-476f-aad7-33eebf09c0bb None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:20.381949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 227/453] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:13:20 2022] GET /identity/v3/projects/6c00a4064cc8477492c9b8b2e5412197 => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:20.387825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cb729-2cde-4558-b1fc-008f249c879e None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:20.490585 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770cb729-2cde-4558-b1fc-008f249c879e None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:20.526588 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451da8b6-b353-4afe-9bd2-c6ccde814cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:20.527128 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451da8b6-b353-4afe-9bd2-c6ccde814cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:20.527466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451da8b6-b353-4afe-9bd2-c6ccde814cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:20.527739 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451da8b6-b353-4afe-9bd2-c6ccde814cd6 None admin] PATH_INFO: `/v3/groups/62f4263724184b8282a082eacc7ac6ac` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:20.528935 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451da8b6-b353-4afe-9bd2-c6ccde814cd6 None admin] RBAC: Authorizing `identity:get_group(group_id=62f4263724184b8282a082eacc7ac6ac)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:20.530392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451da8b6-b353-4afe-9bd2-c6ccde814cd6 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:20.531972 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 227/454] 10.222.0.17 () {56 vars in 1180 bytes} [Sun Aug 7 19:13:20 2022] GET /identity/v3/groups/62f4263724184b8282a082eacc7ac6ac => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:20.538185 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc910af9-a9c8-476f-aad7-33eebf09c0bb None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:20.630742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:20.646758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc910af9-a9c8-476f-aad7-33eebf09c0bb None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:20.680929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d189e4d-cf04-461d-9054-f297dce31f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:20.681549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d189e4d-cf04-461d-9054-f297dce31f63 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:20.681828 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d189e4d-cf04-461d-9054-f297dce31f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:20.682096 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d189e4d-cf04-461d-9054-f297dce31f63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:20.683227 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d189e4d-cf04-461d-9054-f297dce31f63 None admin] RBAC: Adding query filter params (group.id=62f4263724184b8282a082eacc7ac6ac, role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=6c00a4064cc8477492c9b8b2e5412197) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:20.683555 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d189e4d-cf04-461d-9054-f297dce31f63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:20.685031 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d189e4d-cf04-461d-9054-f297dce31f63 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:20.691202 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 228/455] 10.222.0.17 () {56 vars in 1376 bytes} [Sun Aug 7 19:13:20 2022] GET /identity/v3/role_assignments?group.id=62f4263724184b8282a082eacc7ac6ac&scope.project.id=6c00a4064cc8477492c9b8b2e5412197&role.id=4da42f08d107448fb877b42b61bc74ea => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:23.218838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c3621-e8c3-4536-9648-90a347d66bb4 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:23.219361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c3621-e8c3-4536-9648-90a347d66bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:23.219361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c3621-e8c3-4536-9648-90a347d66bb4 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:23.220801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 228/456] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:23.226607 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631decf-8096-4dc7-ae46-657db9f1d543 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:23.227082 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631decf-8096-4dc7-ae46-657db9f1d543 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:23.227154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631decf-8096-4dc7-ae46-657db9f1d543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:23.253078 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-6631decf-8096-4dc7-ae46-657db9f1d543 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:23.254313 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-6631decf-8096-4dc7-ae46-657db9f1d543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:23.346472 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 229/457] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:23.380459 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84134390-99ea-42ac-9fc5-6726c21499b8 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:23.380772 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84134390-99ea-42ac-9fc5-6726c21499b8 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:23.381002 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84134390-99ea-42ac-9fc5-6726c21499b8 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:23.382655 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 229/458] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:23.385596 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631decf-8096-4dc7-ae46-657db9f1d543 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:23.396752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f997291-f30a-445f-ab31-dd2000b2466d None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:23.397053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f997291-f30a-445f-ab31-dd2000b2466d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:23.397300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f997291-f30a-445f-ab31-dd2000b2466d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:23.429050 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-4f997291-f30a-445f-ab31-dd2000b2466d None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:23.430254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4f997291-f30a-445f-ab31-dd2000b2466d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:23.480408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:23.492711 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631decf-8096-4dc7-ae46-657db9f1d543 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:23.511262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:23.524379 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 230/459] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:13:23.525968 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abcb09-31d1-4ed9-9281-4c87eedf626f None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:23.526284 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abcb09-31d1-4ed9-9281-4c87eedf626f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:23.526645 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abcb09-31d1-4ed9-9281-4c87eedf626f None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:23.528406 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 231/460] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:23.532493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1bd93-f0a1-40e9-9920-1cc7a04ada37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:23.533336 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1bd93-f0a1-40e9-9920-1cc7a04ada37 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:23.533532 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c2bf3-906a-4e4b-812d-3e272333e837 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:23.533895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1bd93-f0a1-40e9-9920-1cc7a04ada37 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:23.534003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c2bf3-906a-4e4b-812d-3e272333e837 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:23.534326 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c2bf3-906a-4e4b-812d-3e272333e837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:23.534536 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1bd93-f0a1-40e9-9920-1cc7a04ada37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:23.545704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1bd93-f0a1-40e9-9920-1cc7a04ada37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:23.547413 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1bd93-f0a1-40e9-9920-1cc7a04ada37 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:23.553755 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-7e7c2bf3-906a-4e4b-812d-3e272333e837 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:23.555072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-7e7c2bf3-906a-4e4b-812d-3e272333e837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:23.564866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-f7f1bd93-f0a1-40e9-9920-1cc7a04ada37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:23.564866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:23.564866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:23.564866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:23.564866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:23.564866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:23.564866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:23.564866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:23.564866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:23.564866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:23.564866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:23.564866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:23.564866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:23.564866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:23.564866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:23.564866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:23.565806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:23.565806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:23.565806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:23.565806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:23.565806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:23.565806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:23.565806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:23.565806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:23.565806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:23.565806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:23.565806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:23.565806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:23.565806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:23.565806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:23.565806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:23.565806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:23.565806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:23.566638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:23.566638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:23.566638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:23.569019 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 230/461] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:23.570072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1bd93-f0a1-40e9-9920-1cc7a04ada37 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:23.637363 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 232/462] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:13:23.638593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c2bf3-906a-4e4b-812d-3e272333e837 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:23.717176 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1bd93-f0a1-40e9-9920-1cc7a04ada37 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:23.758593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:23.765240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c2bf3-906a-4e4b-812d-3e272333e837 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:23.767264 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59fe7f-42cd-4563-92e4-6dbade12a4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:23.769170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59fe7f-42cd-4563-92e4-6dbade12a4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:23.769670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59fe7f-42cd-4563-92e4-6dbade12a4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:23.770151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59fe7f-42cd-4563-92e4-6dbade12a4d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:23.785033 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59fe7f-42cd-4563-92e4-6dbade12a4d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:23.787575 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59fe7f-42cd-4563-92e4-6dbade12a4d4 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:23.795024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:23.809174 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765e830-8b86-464e-97de-462f185bd445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:23.809603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-3f59fe7f-42cd-4563-92e4-6dbade12a4d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:23.809603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:23.809603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:23.809603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:23.809603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:23.809603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:23.809603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:23.809603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:23.809603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:23.809603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:23.809603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:23.809603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:23.809603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:23.809603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:23.809603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:23.809603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:23.810490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:23.810490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:23.810490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:23.810490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:23.810490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:23.810490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:23.810490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:23.810490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:23.810490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:23.810490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:23.810490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:23.810490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:23.810490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:23.810490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:23.810490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:23.810490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:23.810490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:23.811343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:23.811343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:23.811343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:23.811511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765e830-8b86-464e-97de-462f185bd445 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:23.811511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765e830-8b86-464e-97de-462f185bd445 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:23.811511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765e830-8b86-464e-97de-462f185bd445 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:23.811511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765e830-8b86-464e-97de-462f185bd445 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:23.811511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765e830-8b86-464e-97de-462f185bd445 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:23.812827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0765e830-8b86-464e-97de-462f185bd445 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:23.814070 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 231/463] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:23.815520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59fe7f-42cd-4563-92e4-6dbade12a4d4 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:23.838205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 233/464] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:23.839247 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765e830-8b86-464e-97de-462f185bd445 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:23.939675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59fe7f-42cd-4563-92e4-6dbade12a4d4 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:23.949307 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765e830-8b86-464e-97de-462f185bd445 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:23.975789 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3ff6c-cae3-4e09-93ec-d6934632e425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:23.976456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3ff6c-cae3-4e09-93ec-d6934632e425 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:23.976735 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3ff6c-cae3-4e09-93ec-d6934632e425 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:23.977228 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3ff6c-cae3-4e09-93ec-d6934632e425 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:23.985071 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a680d43-8af3-458e-9272-8a41b1f14937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:23.985690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a680d43-8af3-458e-9272-8a41b1f14937 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:23.985972 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a680d43-8af3-458e-9272-8a41b1f14937 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:23.986241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a680d43-8af3-458e-9272-8a41b1f14937 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:23.986890 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a680d43-8af3-458e-9272-8a41b1f14937 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:23.987186 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a680d43-8af3-458e-9272-8a41b1f14937 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:23.988630 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a680d43-8af3-458e-9272-8a41b1f14937 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:23.988873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3ff6c-cae3-4e09-93ec-d6934632e425 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:23.990419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3ff6c-cae3-4e09-93ec-d6934632e425 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:24.000370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 234/465] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:24.001175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a680d43-8af3-458e-9272-8a41b1f14937 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:24.018389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-0ff3ff6c-cae3-4e09-93ec-d6934632e425 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:24.018389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:24.018389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:24.018389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:24.018389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:24.018389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:24.018389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:24.018389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:24.018389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:24.018389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:24.018389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:24.018389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:24.018389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:24.018389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:24.018389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:24.018389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:24.019590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:24.019590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:24.019590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:24.019590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:24.019590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:24.019590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:24.019590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:24.019590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:24.019590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:24.019590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:24.019590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:24.019590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:24.019590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:24.019590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:24.019590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:24.019590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:24.019590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:24.021670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:24.021670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:24.021670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:24.021670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 232/466] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:24.021670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e91def-2073-4c01-b4b8-3b9f1911065a None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:24.021670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e91def-2073-4c01-b4b8-3b9f1911065a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:24.021670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e91def-2073-4c01-b4b8-3b9f1911065a None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:24.021670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 233/467] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:24.021670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e91def-2073-4c01-b4b8-3b9f1911065a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:24.056726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:24.062651 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:24.116100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a680d43-8af3-458e-9272-8a41b1f14937 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:24.134576 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e91def-2073-4c01-b4b8-3b9f1911065a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:24.151870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a878c-9d03-40b8-8d4f-b1051c42e8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:24.152589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a878c-9d03-40b8-8d4f-b1051c42e8db None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:24.152870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a878c-9d03-40b8-8d4f-b1051c42e8db None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:24.153433 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a878c-9d03-40b8-8d4f-b1051c42e8db None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:24.154094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a878c-9d03-40b8-8d4f-b1051c42e8db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:24.155566 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a878c-9d03-40b8-8d4f-b1051c42e8db None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:24.170851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805710f-eebf-4f0d-af92-4fee716dac32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:24.171469 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805710f-eebf-4f0d-af92-4fee716dac32 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:24.171742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805710f-eebf-4f0d-af92-4fee716dac32 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:24.172023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805710f-eebf-4f0d-af92-4fee716dac32 None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:24.172719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805710f-eebf-4f0d-af92-4fee716dac32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:24.174239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805710f-eebf-4f0d-af92-4fee716dac32 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:24.184026 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 235/468] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:13:24 2022] POST /identity/v3/users => generated 248 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:13:24.184026 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd0e41-fda2-4b2b-8f80-f49cbf437c39 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:24.184026 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd0e41-fda2-4b2b-8f80-f49cbf437c39 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:24.184026 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd0e41-fda2-4b2b-8f80-f49cbf437c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:24.211223 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-48cd0e41-fda2-4b2b-8f80-f49cbf437c39 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:24.212519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-48cd0e41-fda2-4b2b-8f80-f49cbf437c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:24.216736 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 234/469] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:13:24 2022] POST /identity/v3/users => generated 246 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:13:24.217797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805710f-eebf-4f0d-af92-4fee716dac32 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:24.374836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 236/470] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:24.408492 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd0e41-fda2-4b2b-8f80-f49cbf437c39 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:24.412451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:24.427475 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805710f-eebf-4f0d-af92-4fee716dac32 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:24.443492 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:24.472361 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f16a5-a1ec-4e71-a13d-b287e0060ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:24.472905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f16a5-a1ec-4e71-a13d-b287e0060ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:24.473179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f16a5-a1ec-4e71-a13d-b287e0060ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:24.473913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f16a5-a1ec-4e71-a13d-b287e0060ed6 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:24.474574 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f16a5-a1ec-4e71-a13d-b287e0060ed6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:24.474871 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f16a5-a1ec-4e71-a13d-b287e0060ed6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:24.476854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f16a5-a1ec-4e71-a13d-b287e0060ed6 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:24.491981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 235/471] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:24.499047 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f16a5-a1ec-4e71-a13d-b287e0060ed6 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:24.544604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd0e41-fda2-4b2b-8f80-f49cbf437c39 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:24.584883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571c3c9-ab1a-4704-b6ce-57b632644c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:24.587113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571c3c9-ab1a-4704-b6ce-57b632644c89 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:24.588375 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571c3c9-ab1a-4704-b6ce-57b632644c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:24.588806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f571c3c9-ab1a-4704-b6ce-57b632644c89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:24.611537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571c3c9-ab1a-4704-b6ce-57b632644c89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:24.613321 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f571c3c9-ab1a-4704-b6ce-57b632644c89 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:24.642456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-f571c3c9-ab1a-4704-b6ce-57b632644c89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:24.642456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:24.642456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:24.642456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:24.642456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:24.642456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:24.642456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:24.642456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:24.642456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:24.642456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:24.642456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:24.642456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:24.642456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:24.642456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:24.642456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:24.642456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:24.643596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:24.643596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:24.643596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:24.643596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:24.643596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:24.643596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:24.643596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:24.643596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:24.643596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:24.643596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:24.643596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:24.643596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:24.643596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:24.643596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:24.643596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:24.643596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:24.643596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:24.644681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:24.644681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:24.644681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:24.648986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 237/472] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:24.655821 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571c3c9-ab1a-4704-b6ce-57b632644c89 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:24.679425 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f16a5-a1ec-4e71-a13d-b287e0060ed6 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:24.736840 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d390d-9a12-404b-8bfd-16471c8a9712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:24.737804 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d390d-9a12-404b-8bfd-16471c8a9712 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:24.738128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d390d-9a12-404b-8bfd-16471c8a9712 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:24.738403 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d390d-9a12-404b-8bfd-16471c8a9712 None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:24.739055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d390d-9a12-404b-8bfd-16471c8a9712 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:24.741388 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d390d-9a12-404b-8bfd-16471c8a9712 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:24.771269 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 236/473] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:13:24 2022] POST /identity/v3/users => generated 248 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:13:24.772940 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63189f8-6132-479d-856f-cfe4feb33279 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:24.773226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63189f8-6132-479d-856f-cfe4feb33279 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:24.773496 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63189f8-6132-479d-856f-cfe4feb33279 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:24.774084 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:24.777708 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 237/474] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:24.783813 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a758b5-ae6a-468f-899c-0e35744abdc4 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:24.784145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a758b5-ae6a-468f-899c-0e35744abdc4 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:24.784392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a758b5-ae6a-468f-899c-0e35744abdc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:24.800353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:24.809214 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f571c3c9-ab1a-4704-b6ce-57b632644c89 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:24.810596 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-f9a758b5-ae6a-468f-899c-0e35744abdc4 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:24.812569 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f9a758b5-ae6a-468f-899c-0e35744abdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:24.860590 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf2b13-f40d-4f89-bd44-15ae7ac0b2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:24.863237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf2b13-f40d-4f89-bd44-15ae7ac0b2fc None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:24.864644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf2b13-f40d-4f89-bd44-15ae7ac0b2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:24.867924 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf2b13-f40d-4f89-bd44-15ae7ac0b2fc None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:24.869041 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf2b13-f40d-4f89-bd44-15ae7ac0b2fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:24.870489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf2b13-f40d-4f89-bd44-15ae7ac0b2fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:24.871931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf2b13-f40d-4f89-bd44-15ae7ac0b2fc None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:24.885993 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 238/475] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:24.892192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf2b13-f40d-4f89-bd44-15ae7ac0b2fc None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:24.934358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 238/476] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:24.957642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a758b5-ae6a-468f-899c-0e35744abdc4 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:25.038741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf2b13-f40d-4f89-bd44-15ae7ac0b2fc None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:25.081065 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a2b7c-ce7c-4c64-8460-05ac1a10facf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:25.081693 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a2b7c-ce7c-4c64-8460-05ac1a10facf None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:25.081973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a2b7c-ce7c-4c64-8460-05ac1a10facf None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:25.082249 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a2b7c-ce7c-4c64-8460-05ac1a10facf None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:25.082906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a2b7c-ce7c-4c64-8460-05ac1a10facf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:25.084384 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a2b7c-ce7c-4c64-8460-05ac1a10facf None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:25.097094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:25.097280 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a758b5-ae6a-468f-899c-0e35744abdc4 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:25.112269 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 239/477] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:13:24 2022] POST /identity/v3/users => generated 249 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:13:25.138434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8e0ad-d6e2-4751-877e-6fe5195a1617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:25.139085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8e0ad-d6e2-4751-877e-6fe5195a1617 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:25.139364 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8e0ad-d6e2-4751-877e-6fe5195a1617 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:25.139635 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8e0ad-d6e2-4751-877e-6fe5195a1617 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:25.140377 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8e0ad-d6e2-4751-877e-6fe5195a1617 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:25.141868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8e0ad-d6e2-4751-877e-6fe5195a1617 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:25.146901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:25.152624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.sql.core [None req-d3b8e0ad-d6e2-4751-877e-6fe5195a1617 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 07 19:13:25.152624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 07 19:13:25.152624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [parameters: {'id': '386d8cc2947c497188d44efa61f1e56c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 07 19:13:25.152624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80138) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 07 19:13:25.173324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-d3b8e0ad-d6e2-4751-877e-6fe5195a1617 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 07 19:13:25.173324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:25.173324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 19:13:25.173324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 19:13:25.173324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 19:13:25.173324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 19:13:25.173324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 19:13:25.173324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 19:13:25.173324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 19:13:25.173324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application conn.query(q) Aug 07 19:13:25.173324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 19:13:25.173324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 19:13:25.173324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 19:13:25.173324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application result.read() Aug 07 19:13:25.173324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return ref.to_dict() Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 19:13:25.174276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application next(self.gen) Aug 07 19:13:25.175154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 07 19:13:25.175154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application yield resource Aug 07 19:13:25.175154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 19:13:25.175154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application next(self.gen) Aug 07 19:13:25.175154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 07 19:13:25.175154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self.session.rollback() Aug 07 19:13:25.175154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 07 19:13:25.175154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self.force_reraise() Aug 07 19:13:25.175154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 07 19:13:25.175154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise self.value Aug 07 19:13:25.175154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 07 19:13:25.175154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 07 19:13:25.175154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 07 19:13:25.175154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application session.commit() Aug 07 19:13:25.175154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 07 19:13:25.175154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 07 19:13:25.175154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 07 19:13:25.176122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._prepare_impl() Aug 07 19:13:25.176122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 07 19:13:25.176122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self.session.flush() Aug 07 19:13:25.176122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 07 19:13:25.176122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._flush(objects) Aug 07 19:13:25.176122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 07 19:13:25.176122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 07 19:13:25.176122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 07 19:13:25.176122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application compat.raise_( Aug 07 19:13:25.176122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 19:13:25.176122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception Aug 07 19:13:25.176122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 07 19:13:25.176122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application flush_context.execute() Aug 07 19:13:25.176122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 07 19:13:25.176122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rec.execute(self) Aug 07 19:13:25.176122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 07 19:13:25.176122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 07 19:13:25.177000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 07 19:13:25.177000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 07 19:13:25.177000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 07 19:13:25.177000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 07 19:13:25.177000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 07 19:13:25.177000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 07 19:13:25.177000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 07 19:13:25.177000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 07 19:13:25.177000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 07 19:13:25.177000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 07 19:13:25.177000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 07 19:13:25.177000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 07 19:13:25.177000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 07 19:13:25.177000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 07 19:13:25.177000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 19:13:25.177000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception Aug 07 19:13:25.178741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 19:13:25.178741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 19:13:25.178741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 19:13:25.178741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 19:13:25.178741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 19:13:25.178741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 19:13:25.178741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 19:13:25.178741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application conn.query(q) Aug 07 19:13:25.178741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 19:13:25.178741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 19:13:25.178741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 19:13:25.178741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application result.read() Aug 07 19:13:25.178741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 19:13:25.178741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 19:13:25.178741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 19:13:25.178741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 19:13:25.180654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 19:13:25.180654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 19:13:25.180654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 19:13:25.180654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 19:13:25.180654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 07 19:13:25.180654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 07 19:13:25.180654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [parameters: {'id': '386d8cc2947c497188d44efa61f1e56c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 07 19:13:25.180654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 07 19:13:25.180654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:25.180654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:13:25.180654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:25.180654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:25.180654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:25.180654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:25.180654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:25.180654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:25.182832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:25.182832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:25.182832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:25.182832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:25.182832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:25.182832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:25.182832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 07 19:13:25.182832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 07 19:13:25.182832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:25.182832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:25.182832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 07 19:13:25.182832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 07 19:13:25.182832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 07 19:13:25.182832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 07 19:13:25.182832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 07 19:13:25.182832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:25.184803 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 239/478] 10.222.0.17 () {60 vars in 1131 bytes} [Sun Aug 7 19:13:24 2022] POST /identity/v3/roles => generated 144 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:25.184932 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a2b7c-ce7c-4c64-8460-05ac1a10facf None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:25.315048 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a2b7c-ce7c-4c64-8460-05ac1a10facf None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:25.350394 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdfd0e-d04b-4e7b-876e-330ef674e544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:25.351044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdfd0e-d04b-4e7b-876e-330ef674e544 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:25.351331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdfd0e-d04b-4e7b-876e-330ef674e544 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:25.351604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdfd0e-d04b-4e7b-876e-330ef674e544 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:25.362224 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdfd0e-d04b-4e7b-876e-330ef674e544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:25.367424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-36fdfd0e-d04b-4e7b-876e-330ef674e544 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:13:25.367424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:25.367424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:25.367424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:25.367424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:25.367424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:25.367424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:25.367424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:25.367424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:25.367424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:25.367424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:25.367424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:25.367424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:25.367424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:25.367424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:25.367424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:25.369438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:25.369438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:25.369438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:25.369438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:25.369438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:25.369438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:25.369438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:25.369438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:25.369438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:25.369438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:25.369438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:25.369438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:25.369438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:25.369438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:25.369438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:25.369438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:25.371346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:25.371346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:25.371346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:25.371346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:25.371346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:25.371346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:25.371346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:25.371346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:25.371346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:25.371346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:25.371346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 19:13:25.371346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:25.371346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 240/479] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:13:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:25.375719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8e0ad-d6e2-4751-877e-6fe5195a1617 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:25.487295 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8e0ad-d6e2-4751-877e-6fe5195a1617 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:25.524493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c3ce9-53c1-4933-98aa-e88bff0ac13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:25.525120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c3ce9-53c1-4933-98aa-e88bff0ac13d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:25.525437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c3ce9-53c1-4933-98aa-e88bff0ac13d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:25.525718 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c3ce9-53c1-4933-98aa-e88bff0ac13d None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:25.526403 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c3ce9-53c1-4933-98aa-e88bff0ac13d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:25.526754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c3ce9-53c1-4933-98aa-e88bff0ac13d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:25.528312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c3ce9-53c1-4933-98aa-e88bff0ac13d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:25.537557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 240/480] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:26.740056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5def879-8057-4ff6-a423-c7dc9714d575 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:26.740056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5def879-8057-4ff6-a423-c7dc9714d575 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:26.740056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5def879-8057-4ff6-a423-c7dc9714d575 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:26.741268 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 241/481] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:26.749983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bcc4d-5f19-4a1b-9c90-02a4704dcacf None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:26.749983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bcc4d-5f19-4a1b-9c90-02a4704dcacf None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:26.749983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bcc4d-5f19-4a1b-9c90-02a4704dcacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:26.772978 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-b21bcc4d-5f19-4a1b-9c90-02a4704dcacf None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:26.774514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b21bcc4d-5f19-4a1b-9c90-02a4704dcacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:26.859555 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:26.870688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 241/482] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:26.923518 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5def879-8057-4ff6-a423-c7dc9714d575 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:26.999817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:27.047386 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5def879-8057-4ff6-a423-c7dc9714d575 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:27.086231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878903e-ffd4-4690-8541-67439f4cb928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:27.086892 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878903e-ffd4-4690-8541-67439f4cb928 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:27.087177 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878903e-ffd4-4690-8541-67439f4cb928 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:27.087456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878903e-ffd4-4690-8541-67439f4cb928 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:27.106243 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878903e-ffd4-4690-8541-67439f4cb928 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:27.111827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-c878903e-ffd4-4690-8541-67439f4cb928 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:27.111827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:27.111827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:27.111827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:27.111827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:27.111827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:27.111827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:27.111827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:27.111827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:27.111827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:27.111827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:27.111827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:27.111827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:27.111827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:27.111827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:27.111827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:27.112736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:27.112736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:27.112736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:27.112736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:27.112736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:27.112736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:27.112736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:27.112736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:27.112736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:27.112736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:27.112736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:27.112736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:27.112736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:27.112736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:27.112736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:27.112736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:27.113524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:27.113524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:27.113524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:27.113524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:27.113524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:27.113524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:27.113524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:27.113524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:27.113524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:27.113524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:27.113524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:27.113524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:27.116106 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 242/483] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:27.121917 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bcc4d-5f19-4a1b-9c90-02a4704dcacf None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:27.231074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bcc4d-5f19-4a1b-9c90-02a4704dcacf None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:27.282330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049689f1-a1ba-40c3-b55c-1f4c6e7435d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:27.282877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049689f1-a1ba-40c3-b55c-1f4c6e7435d3 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:27.283149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049689f1-a1ba-40c3-b55c-1f4c6e7435d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:27.283412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049689f1-a1ba-40c3-b55c-1f4c6e7435d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:27.284132 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049689f1-a1ba-40c3-b55c-1f4c6e7435d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:27.284437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049689f1-a1ba-40c3-b55c-1f4c6e7435d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:27.285825 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049689f1-a1ba-40c3-b55c-1f4c6e7435d3 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:27.299834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 242/484] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:13:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:27.305131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878903e-ffd4-4690-8541-67439f4cb928 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:27.378621 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61081b2-6648-46cd-8e7d-fefe1a15a2bc None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:27.378949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61081b2-6648-46cd-8e7d-fefe1a15a2bc None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:27.379166 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61081b2-6648-46cd-8e7d-fefe1a15a2bc None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:27.380821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 243/485] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:27.386859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595eb7c-d723-4771-855e-ae2b5c34b9c5 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:27.387402 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595eb7c-d723-4771-855e-ae2b5c34b9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:27.388426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595eb7c-d723-4771-855e-ae2b5c34b9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:27.409088 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-1595eb7c-d723-4771-855e-ae2b5c34b9c5 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:27.410725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-1595eb7c-d723-4771-855e-ae2b5c34b9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:27.488789 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878903e-ffd4-4690-8541-67439f4cb928 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:27.507071 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:27.509895 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:27.529926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20637d-e34c-49a7-b00a-90ddede398ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:27.530953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20637d-e34c-49a7-b00a-90ddede398ce None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:27.531431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20637d-e34c-49a7-b00a-90ddede398ce None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:27.531910 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20637d-e34c-49a7-b00a-90ddede398ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:27.539113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 244/486] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:13:27.539113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a916c-7296-4a0b-a3a0-cb242ecd48e4 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:27.539113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a916c-7296-4a0b-a3a0-cb242ecd48e4 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:27.539113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a916c-7296-4a0b-a3a0-cb242ecd48e4 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:27.539113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 245/487] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:27.541066 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bb2de-a8ec-4431-a814-d7fc669ea41c None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:27.541350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bb2de-a8ec-4431-a814-d7fc669ea41c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:27.541935 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bb2de-a8ec-4431-a814-d7fc669ea41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:27.560506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20637d-e34c-49a7-b00a-90ddede398ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:27.562340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-5a7bb2de-a8ec-4431-a814-d7fc669ea41c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:27.562455 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20637d-e34c-49a7-b00a-90ddede398ce None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:27.563742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-5a7bb2de-a8ec-4431-a814-d7fc669ea41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:27.584615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-cd20637d-e34c-49a7-b00a-90ddede398ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:27.584615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:27.584615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:27.584615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:27.584615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:27.584615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:27.584615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:27.584615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:27.584615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:27.584615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:27.584615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:27.584615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:27.584615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:27.584615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:27.584615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:27.584615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:27.585591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:27.585591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:27.585591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:27.585591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:27.585591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:27.585591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:27.585591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:27.585591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:27.585591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:27.585591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:27.585591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:27.585591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:27.585591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:27.585591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:27.585591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:27.585591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:27.585591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:27.586438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:27.586438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:27.586438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:27.586438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 243/488] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:27.586647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20637d-e34c-49a7-b00a-90ddede398ce None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:27.659662 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 246/489] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:13:27.660936 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bb2de-a8ec-4431-a814-d7fc669ea41c None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:27.725493 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20637d-e34c-49a7-b00a-90ddede398ce None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:27.765364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9bcbb-a392-45e7-a712-e3b75bd278c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:27.766023 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9bcbb-a392-45e7-a712-e3b75bd278c3 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:27.766243 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9bcbb-a392-45e7-a712-e3b75bd278c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:27.766414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9bcbb-a392-45e7-a712-e3b75bd278c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:27.775405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bb2de-a8ec-4431-a814-d7fc669ea41c None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:27.778589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9bcbb-a392-45e7-a712-e3b75bd278c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:27.785056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-59d9bcbb-a392-45e7-a712-e3b75bd278c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:27.785056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:27.785056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:27.785056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:27.785056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:27.785056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:27.785056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:27.785056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:27.785056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:27.785056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:27.785056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:27.785056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:27.785056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:27.785056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:27.785056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:27.785056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:27.786133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:27.786133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:27.786133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:27.786133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:27.786133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:27.786133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:27.786133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:27.786133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:27.786133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:27.786133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:27.786133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:27.786133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:27.786133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:27.786133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:27.786133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:27.786133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:27.786929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:27.786929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:27.786929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:27.786929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:27.786929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:27.786929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:27.786929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:27.786929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:27.786929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:27.786929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:27.786929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:27.786929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:27.786929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 244/490] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:27.787583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9bcbb-a392-45e7-a712-e3b75bd278c3 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:27.795780 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:27.818609 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3de7d-e8b7-4a81-ad45-c1a78e5176d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:27.819592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3de7d-e8b7-4a81-ad45-c1a78e5176d7 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:27.820156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3de7d-e8b7-4a81-ad45-c1a78e5176d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:27.820623 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3de7d-e8b7-4a81-ad45-c1a78e5176d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:27.821477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3de7d-e8b7-4a81-ad45-c1a78e5176d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:27.822002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3de7d-e8b7-4a81-ad45-c1a78e5176d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:27.823583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3de7d-e8b7-4a81-ad45-c1a78e5176d7 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:27.825452 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:27.836437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 247/491] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:27.842056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3de7d-e8b7-4a81-ad45-c1a78e5176d7 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:27.906109 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9bcbb-a392-45e7-a712-e3b75bd278c3 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:27.943256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856cf87-644b-4e60-9371-45f5245062ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:27.944150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856cf87-644b-4e60-9371-45f5245062ea None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:27.944411 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856cf87-644b-4e60-9371-45f5245062ea None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:27.944702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856cf87-644b-4e60-9371-45f5245062ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:27.946605 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3de7d-e8b7-4a81-ad45-c1a78e5176d7 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:27.955837 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856cf87-644b-4e60-9371-45f5245062ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:27.961506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-b856cf87-644b-4e60-9371-45f5245062ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:27.961506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:27.961506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:27.961506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:27.961506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:27.961506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:27.961506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:27.961506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:27.961506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:27.961506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:27.961506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:27.961506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:27.961506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:27.961506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:27.961506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:27.961506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:27.962400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:27.962400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:27.962400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:27.962400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:27.962400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:27.962400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:27.962400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:27.962400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:27.962400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:27.962400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:27.962400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:27.962400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:27.962400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:27.962400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:27.962400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:27.962400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:27.963238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:27.963238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:27.963238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:27.963238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:27.963238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:27.963238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:27.963238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:27.963238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:27.963238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:27.963238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:27.963238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:27.963238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:27.967535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 245/492] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:27.968581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856cf87-644b-4e60-9371-45f5245062ea None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:27.986126 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e635109-e792-4c3a-b5d1-9ddb0b38650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:27.987112 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e635109-e792-4c3a-b5d1-9ddb0b38650f None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:27.987589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e635109-e792-4c3a-b5d1-9ddb0b38650f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:27.988113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e635109-e792-4c3a-b5d1-9ddb0b38650f None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:27.988982 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e635109-e792-4c3a-b5d1-9ddb0b38650f None admin] RBAC: Adding query filter params (name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:27.989481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e635109-e792-4c3a-b5d1-9ddb0b38650f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:27.991096 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e635109-e792-4c3a-b5d1-9ddb0b38650f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:28.000535 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 248/493] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:13:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:28.001685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e635109-e792-4c3a-b5d1-9ddb0b38650f None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:28.056399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:28.083511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:28.098290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856cf87-644b-4e60-9371-45f5245062ea None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:28.134999 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e635109-e792-4c3a-b5d1-9ddb0b38650f None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:28.136174 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0397c46-2ffb-4590-81b6-3d50af2946d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:28.136836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0397c46-2ffb-4590-81b6-3d50af2946d5 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:28.137113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0397c46-2ffb-4590-81b6-3d50af2946d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:28.137379 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0397c46-2ffb-4590-81b6-3d50af2946d5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:28.144941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0397c46-2ffb-4590-81b6-3d50af2946d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:28.146401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0397c46-2ffb-4590-81b6-3d50af2946d5 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:28.159905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-e0397c46-2ffb-4590-81b6-3d50af2946d5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 07 19:13:28.159905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:28.159905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:28.159905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:28.159905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:28.159905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:28.159905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:28.159905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:28.159905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:28.159905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:28.159905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:28.159905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:28.159905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:28.159905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:28.159905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:28.159905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:28.161017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:28.162126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:28.162126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:28.162126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:28.162126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:28.162126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:28.162126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:28.162126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:28.162126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:28.162126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 07 19:13:28.162126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:28.163917 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 246/494] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:28.164875 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0397c46-2ffb-4590-81b6-3d50af2946d5 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:28.176905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0bd72-cc75-48e3-8d34-c0297ab72b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:28.177948 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0bd72-cc75-48e3-8d34-c0297ab72b40 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:28.178366 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0bd72-cc75-48e3-8d34-c0297ab72b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:28.178775 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0bd72-cc75-48e3-8d34-c0297ab72b40 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:28.179585 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0bd72-cc75-48e3-8d34-c0297ab72b40 None admin] RBAC: Adding query filter params (name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:28.180143 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0bd72-cc75-48e3-8d34-c0297ab72b40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:28.181867 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0bd72-cc75-48e3-8d34-c0297ab72b40 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:28.196086 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 249/495] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:13:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:28.197746 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d724819-64dd-4bb9-853c-9fd4b43d6c92 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:28.198298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d724819-64dd-4bb9-853c-9fd4b43d6c92 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:28.198711 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d724819-64dd-4bb9-853c-9fd4b43d6c92 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:28.200822 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 250/496] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:28.201550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d724819-64dd-4bb9-853c-9fd4b43d6c92 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:28.268010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0397c46-2ffb-4590-81b6-3d50af2946d5 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:28.301675 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d79b20-39cb-4763-b6ad-6dae74ec8c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:28.302302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d79b20-39cb-4763-b6ad-6dae74ec8c20 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:28.302582 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d79b20-39cb-4763-b6ad-6dae74ec8c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:28.302849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d79b20-39cb-4763-b6ad-6dae74ec8c20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:28.303564 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:28.309205 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d724819-64dd-4bb9-853c-9fd4b43d6c92 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:28.315297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d79b20-39cb-4763-b6ad-6dae74ec8c20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:28.316795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d79b20-39cb-4763-b6ad-6dae74ec8c20 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:28.325004 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:28.332503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-89d79b20-39cb-4763-b6ad-6dae74ec8c20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:28.332503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:28.332503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:28.332503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:28.332503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:28.332503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:28.332503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:28.332503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:28.332503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:28.332503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:28.332503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:28.332503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:28.332503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:28.332503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:28.332503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:28.332503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:28.333491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:28.333491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:28.333491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:28.333491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:28.333491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:28.333491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:28.333491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:28.333491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:28.333491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:28.333491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:28.333491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:28.333491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:28.333491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:28.333491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:28.333491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:28.333491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:28.333491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:28.334307 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:28.334307 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:28.334307 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:28.334307 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 247/497] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:28.335368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d79b20-39cb-4763-b6ad-6dae74ec8c20 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:28.347520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53d685-28c5-41da-9f11-4bb2f292a923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:28.348201 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53d685-28c5-41da-9f11-4bb2f292a923 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:28.348474 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53d685-28c5-41da-9f11-4bb2f292a923 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:28.349003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53d685-28c5-41da-9f11-4bb2f292a923 None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:28.349500 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53d685-28c5-41da-9f11-4bb2f292a923 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:28.349770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53d685-28c5-41da-9f11-4bb2f292a923 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:28.351272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53d685-28c5-41da-9f11-4bb2f292a923 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:28.390275 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 251/498] 10.222.0.17 () {56 vars in 1137 bytes} [Sun Aug 7 19:13:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:28.391567 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e0153-b3fe-4389-99e4-1900e381c8be None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:28.391852 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e0153-b3fe-4389-99e4-1900e381c8be None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:28.392156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e0153-b3fe-4389-99e4-1900e381c8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:28.411143 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-993e0153-b3fe-4389-99e4-1900e381c8be None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:28.412397 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-993e0153-b3fe-4389-99e4-1900e381c8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:28.443434 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d79b20-39cb-4763-b6ad-6dae74ec8c20 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:28.480362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ea13e-a6f4-4a96-b70a-d87a8a34aa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:28.481079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ea13e-a6f4-4a96-b70a-d87a8a34aa89 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:28.481448 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ea13e-a6f4-4a96-b70a-d87a8a34aa89 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:28.481819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ea13e-a6f4-4a96-b70a-d87a8a34aa89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:28.483468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 252/499] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:28.484805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c672c-e881-4987-a412-63378493c5ba None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:28.485085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c672c-e881-4987-a412-63378493c5ba None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:28.485356 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c672c-e881-4987-a412-63378493c5ba None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:28.486742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 253/500] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:28.487253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c672c-e881-4987-a412-63378493c5ba None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:28.507063 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ea13e-a6f4-4a96-b70a-d87a8a34aa89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:28.508606 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ea13e-a6f4-4a96-b70a-d87a8a34aa89 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:28.525240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-6c2ea13e-a6f4-4a96-b70a-d87a8a34aa89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:28.525240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:28.525240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:28.525240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:28.525240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:28.525240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:28.525240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:28.525240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:28.525240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:28.525240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:28.525240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:28.525240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:28.525240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:28.525240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:28.525240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:28.525240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:28.526401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:28.526401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:28.526401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:28.526401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:28.526401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:28.526401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:28.526401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:28.526401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:28.526401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:28.526401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:28.526401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:28.526401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:28.526401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:28.526401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:28.526401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:28.526401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:28.526401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:28.527720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:28.527720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:28.527720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:28.527720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 248/501] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:28.527720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ea13e-a6f4-4a96-b70a-d87a8a34aa89 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:28.582503 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:28.584455 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:28.600226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c672c-e881-4987-a412-63378493c5ba None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:28.635771 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d7e53-f730-4f80-965f-7c3f3159f9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:28.636416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d7e53-f730-4f80-965f-7c3f3159f9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:28.636730 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d7e53-f730-4f80-965f-7c3f3159f9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:28.637016 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d7e53-f730-4f80-965f-7c3f3159f9e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:28.637664 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d7e53-f730-4f80-965f-7c3f3159f9e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:28.637964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d7e53-f730-4f80-965f-7c3f3159f9e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:28.638305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ea13e-a6f4-4a96-b70a-d87a8a34aa89 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:28.639352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d7e53-f730-4f80-965f-7c3f3159f9e5 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:28.651114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 254/502] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:28.652644 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5e2b8-9331-4b39-8874-dd8d6353476a None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:28.652930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5e2b8-9331-4b39-8874-dd8d6353476a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:28.653205 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5e2b8-9331-4b39-8874-dd8d6353476a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:28.671823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-ffe5e2b8-9331-4b39-8874-dd8d6353476a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:28.673065 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-ffe5e2b8-9331-4b39-8874-dd8d6353476a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:28.678208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1585209-50c1-4a78-b90e-7b0bb7c6f4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:28.678823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1585209-50c1-4a78-b90e-7b0bb7c6f4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:28.679100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1585209-50c1-4a78-b90e-7b0bb7c6f4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:28.679369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1585209-50c1-4a78-b90e-7b0bb7c6f4b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:28.743030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 255/503] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:28.743908 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5e2b8-9331-4b39-8874-dd8d6353476a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:28.748673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1585209-50c1-4a78-b90e-7b0bb7c6f4b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:28.750153 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1585209-50c1-4a78-b90e-7b0bb7c6f4b4 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:28.803460 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:28.828422 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:28.851331 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5e2b8-9331-4b39-8874-dd8d6353476a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:28.852230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 249/504] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:28.853090 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1585209-50c1-4a78-b90e-7b0bb7c6f4b4 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:28.886700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264212d-2378-48ce-a61e-b763e39d79fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:28.887330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264212d-2378-48ce-a61e-b763e39d79fb None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:28.887604 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264212d-2378-48ce-a61e-b763e39d79fb None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:28.887869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264212d-2378-48ce-a61e-b763e39d79fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:28.898711 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264212d-2378-48ce-a61e-b763e39d79fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:28.903510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-3264212d-2378-48ce-a61e-b763e39d79fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:28.903510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:28.903510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:28.903510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:28.903510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:28.903510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:28.903510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:28.903510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:28.903510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:28.903510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:28.903510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:28.903510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:28.903510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:28.903510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:28.903510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:28.903510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:28.904493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:28.904493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:28.904493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:28.904493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:28.904493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:28.904493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:28.904493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:28.904493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:28.904493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:28.904493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:28.904493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:28.904493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:28.904493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:28.904493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:28.904493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:28.904493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:28.905289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:28.905289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:28.905289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:28.905289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:28.905289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:28.905289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:28.905289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:28.905289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:28.905289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:28.905289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:28.905289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:28.905289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:28.905289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 256/505] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:28.906430 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264212d-2378-48ce-a61e-b763e39d79fb None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:28.958974 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1585209-50c1-4a78-b90e-7b0bb7c6f4b4 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:28.994053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb38840-6a21-4cf0-8cb2-96841a5981d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:28.994667 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb38840-6a21-4cf0-8cb2-96841a5981d8 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:28.994956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb38840-6a21-4cf0-8cb2-96841a5981d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:28.995225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb38840-6a21-4cf0-8cb2-96841a5981d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:28.996046 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb38840-6a21-4cf0-8cb2-96841a5981d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:28.996253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb38840-6a21-4cf0-8cb2-96841a5981d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:28.997681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb38840-6a21-4cf0-8cb2-96841a5981d8 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:29.014219 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 250/506] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:29.014522 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264212d-2378-48ce-a61e-b763e39d79fb None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:29.014638 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb38840-6a21-4cf0-8cb2-96841a5981d8 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:29.026265 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:29.050120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70961540-f1d1-491f-9ebd-d2bc209c2013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:29.050789 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70961540-f1d1-491f-9ebd-d2bc209c2013 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:29.051067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70961540-f1d1-491f-9ebd-d2bc209c2013 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:29.051351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70961540-f1d1-491f-9ebd-d2bc209c2013 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:29.054427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:29.060439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70961540-f1d1-491f-9ebd-d2bc209c2013 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:29.062104 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70961540-f1d1-491f-9ebd-d2bc209c2013 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:29.076079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-70961540-f1d1-491f-9ebd-d2bc209c2013 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 19:13:29.076079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:29.076079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:29.076079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:29.076079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:29.076079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:29.076079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:29.076079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:29.076079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:29.076079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:29.076079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:29.076079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:29.076079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:29.076079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:29.076079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:29.076079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:29.077080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:29.077972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:29.077972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:29.077972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:29.077972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:29.077972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:29.077972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:29.077972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:29.077972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:29.077972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 19:13:29.077972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:29.077972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 257/507] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:29.078541 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70961540-f1d1-491f-9ebd-d2bc209c2013 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:29.119260 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb38840-6a21-4cf0-8cb2-96841a5981d8 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:29.153239 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefee00-dfee-4f01-9aa0-e923f8d619b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:29.153892 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefee00-dfee-4f01-9aa0-e923f8d619b3 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:29.154171 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefee00-dfee-4f01-9aa0-e923f8d619b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:29.154443 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefee00-dfee-4f01-9aa0-e923f8d619b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:29.165839 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefee00-dfee-4f01-9aa0-e923f8d619b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:29.167326 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefee00-dfee-4f01-9aa0-e923f8d619b3 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:29.181785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-6aefee00-dfee-4f01-9aa0-e923f8d619b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:29.181785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:29.181785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:29.181785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:29.181785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:29.181785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:29.181785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:29.181785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:29.181785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:29.181785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:29.181785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:29.181785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:29.181785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:29.181785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:29.181785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:29.181785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:29.183118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:29.183118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:29.183118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:29.183118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:29.183118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:29.183118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:29.183118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:29.183118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:29.183118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:29.183118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:29.183118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:29.183118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:29.183118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:29.183118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:29.183118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:29.183118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:29.183118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:29.184188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:29.184188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:29.184188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:29.184188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 251/508] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:29.184188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefee00-dfee-4f01-9aa0-e923f8d619b3 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:29.190986 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70961540-f1d1-491f-9ebd-d2bc209c2013 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:29.225419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852684db-8310-443c-8f61-b173e4e30373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:29.226058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852684db-8310-443c-8f61-b173e4e30373 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:29.226336 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852684db-8310-443c-8f61-b173e4e30373 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:29.226603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852684db-8310-443c-8f61-b173e4e30373 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:29.237628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852684db-8310-443c-8f61-b173e4e30373 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:29.239127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852684db-8310-443c-8f61-b173e4e30373 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:29.248644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:29.254210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-852684db-8310-443c-8f61-b173e4e30373 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:29.254210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:29.254210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:29.254210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:29.254210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:29.254210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:29.254210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:29.254210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:29.254210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:29.254210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:29.254210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:29.254210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:29.254210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:29.254210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:29.254210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:29.254210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:29.255153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:29.255153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:29.255153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:29.255153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:29.255153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:29.255153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:29.255153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:29.255153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:29.255153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:29.255153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:29.255153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:29.255153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:29.255153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:29.255153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:29.255153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:29.255153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:29.255153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:29.256041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:29.256041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:29.256041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:29.256041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 258/509] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:29.256276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852684db-8310-443c-8f61-b173e4e30373 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:29.298168 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefee00-dfee-4f01-9aa0-e923f8d619b3 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:29.318576 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:29.337572 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93381534-3da7-4cb7-8483-9b321eb19de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:29.338192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93381534-3da7-4cb7-8483-9b321eb19de4 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:29.338474 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93381534-3da7-4cb7-8483-9b321eb19de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:29.338747 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93381534-3da7-4cb7-8483-9b321eb19de4 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:29.339545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93381534-3da7-4cb7-8483-9b321eb19de4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:29.340148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93381534-3da7-4cb7-8483-9b321eb19de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:29.341543 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93381534-3da7-4cb7-8483-9b321eb19de4 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:29.350577 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 252/510] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:13:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:29.351446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93381534-3da7-4cb7-8483-9b321eb19de4 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:29.389053 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852684db-8310-443c-8f61-b173e4e30373 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:29.424575 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d5a54-e7fe-4be5-956d-7b7c34e67dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:29.425211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d5a54-e7fe-4be5-956d-7b7c34e67dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:29.425486 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d5a54-e7fe-4be5-956d-7b7c34e67dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:29.425740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d5a54-e7fe-4be5-956d-7b7c34e67dd8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:29.433801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d5a54-e7fe-4be5-956d-7b7c34e67dd8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:29.435278 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d5a54-e7fe-4be5-956d-7b7c34e67dd8 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:29.446072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-191d5a54-e7fe-4be5-956d-7b7c34e67dd8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:13:29.446072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:29.446072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:29.446072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:29.446072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:29.446072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:29.446072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:29.446072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:29.446072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:29.446072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:29.446072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:29.446072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:29.446072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:29.446072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:29.446072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:29.446072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:29.447149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:29.448107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:29.448107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:29.448107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:29.448107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:29.448107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:29.448107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:29.448107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:29.448107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:29.448107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:13:29.448107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:29.448107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 259/511] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Aug 7 19:13:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:29.448657 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d5a54-e7fe-4be5-956d-7b7c34e67dd8 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:29.465159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93381534-3da7-4cb7-8483-9b321eb19de4 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:29.490102 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:29.499260 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282b481-6b79-473f-a5bd-6ce5af186072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:29.499871 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282b481-6b79-473f-a5bd-6ce5af186072 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:29.500206 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282b481-6b79-473f-a5bd-6ce5af186072 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:29.500480 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d282b481-6b79-473f-a5bd-6ce5af186072 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:29.501196 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282b481-6b79-473f-a5bd-6ce5af186072 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:29.501519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282b481-6b79-473f-a5bd-6ce5af186072 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:29.502955 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d282b481-6b79-473f-a5bd-6ce5af186072 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:29.538848 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 253/512] 10.222.0.17 () {56 vars in 1137 bytes} [Sun Aug 7 19:13:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:29.539693 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282b481-6b79-473f-a5bd-6ce5af186072 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:29.548613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d5a54-e7fe-4be5-956d-7b7c34e67dd8 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:29.558792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:29.583931 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2e5eb-066a-49c5-ac3d-63e488df5a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:29.584576 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2e5eb-066a-49c5-ac3d-63e488df5a34 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:29.584848 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2e5eb-066a-49c5-ac3d-63e488df5a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:29.585133 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2e5eb-066a-49c5-ac3d-63e488df5a34 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:29.585778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2e5eb-066a-49c5-ac3d-63e488df5a34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:29.586080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2e5eb-066a-49c5-ac3d-63e488df5a34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:29.587461 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2e5eb-066a-49c5-ac3d-63e488df5a34 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:29.599410 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 260/513] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:29.600319 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2e5eb-066a-49c5-ac3d-63e488df5a34 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:29.646622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d282b481-6b79-473f-a5bd-6ce5af186072 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:29.683009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b394c92-f109-44ac-a887-ebaf5ee38376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:29.683660 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b394c92-f109-44ac-a887-ebaf5ee38376 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:29.683937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b394c92-f109-44ac-a887-ebaf5ee38376 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:29.684240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b394c92-f109-44ac-a887-ebaf5ee38376 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:29.684882 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b394c92-f109-44ac-a887-ebaf5ee38376 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:29.685176 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b394c92-f109-44ac-a887-ebaf5ee38376 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:29.686556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b394c92-f109-44ac-a887-ebaf5ee38376 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:29.697573 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 254/514] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:29.698399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b394c92-f109-44ac-a887-ebaf5ee38376 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:29.699410 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2e5eb-066a-49c5-ac3d-63e488df5a34 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:29.735052 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19cbb3-e1fe-4575-a2cc-3b5b9570be5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:29.735792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19cbb3-e1fe-4575-a2cc-3b5b9570be5c None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:29.736160 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19cbb3-e1fe-4575-a2cc-3b5b9570be5c None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:29.736491 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19cbb3-e1fe-4575-a2cc-3b5b9570be5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:29.744358 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:29.747195 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19cbb3-e1fe-4575-a2cc-3b5b9570be5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:29.748709 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19cbb3-e1fe-4575-a2cc-3b5b9570be5c None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:29.763329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-2e19cbb3-e1fe-4575-a2cc-3b5b9570be5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:29.763329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:29.763329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:29.763329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:29.763329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:29.763329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:29.763329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:29.763329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:29.763329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:29.763329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:29.763329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:29.763329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:29.763329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:29.763329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:29.763329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:29.763329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:29.764420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:29.764420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:29.764420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:29.764420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:29.764420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:29.764420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:29.764420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:29.764420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:29.764420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:29.764420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:29.764420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:29.764420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:29.764420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:29.764420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:29.764420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:29.764420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:29.764420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:29.765477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:29.765477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:29.765477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:29.765477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 261/515] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:29.765477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19cbb3-e1fe-4575-a2cc-3b5b9570be5c None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:29.791325 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:29.806000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b394c92-f109-44ac-a887-ebaf5ee38376 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:29.841940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd10c7-3c94-4f7b-b742-630f0c14b96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:29.842553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd10c7-3c94-4f7b-b742-630f0c14b96e None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:29.842827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd10c7-3c94-4f7b-b742-630f0c14b96e None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:29.843092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd10c7-3c94-4f7b-b742-630f0c14b96e None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:29.843819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd10c7-3c94-4f7b-b742-630f0c14b96e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:29.844150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd10c7-3c94-4f7b-b742-630f0c14b96e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:29.845573 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd10c7-3c94-4f7b-b742-630f0c14b96e None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:29.864441 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 255/516] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Aug 7 19:13:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:29.865778 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd10c7-3c94-4f7b-b742-630f0c14b96e None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:29.881806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19cbb3-e1fe-4575-a2cc-3b5b9570be5c None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:29.916049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41596e-0c9c-4142-a9e1-effe6ba3a4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:29.916668 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41596e-0c9c-4142-a9e1-effe6ba3a4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:29.916941 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41596e-0c9c-4142-a9e1-effe6ba3a4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:29.917208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41596e-0c9c-4142-a9e1-effe6ba3a4a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:29.964339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd10c7-3c94-4f7b-b742-630f0c14b96e None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:29.972204 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:29.984283 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41596e-0c9c-4142-a9e1-effe6ba3a4a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:29.985797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f41596e-0c9c-4142-a9e1-effe6ba3a4a3 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:30.002280 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbedfef-e9d0-4bb5-841a-4153a088af0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:30.002901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbedfef-e9d0-4bb5-841a-4153a088af0d None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:30.003170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbedfef-e9d0-4bb5-841a-4153a088af0d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:30.003446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbedfef-e9d0-4bb5-841a-4153a088af0d None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:30.004119 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbedfef-e9d0-4bb5-841a-4153a088af0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:30.005571 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbedfef-e9d0-4bb5-841a-4153a088af0d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:30.006103 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:30.026339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 256/517] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:13:29 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:13:30.027244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbedfef-e9d0-4bb5-841a-4153a088af0d None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:30.114354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 262/518] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:30.115357 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41596e-0c9c-4142-a9e1-effe6ba3a4a3 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:30.155244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbedfef-e9d0-4bb5-841a-4153a088af0d None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:30.197002 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c797a-ec4b-4bab-a1ac-579a22ee725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:30.197580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c797a-ec4b-4bab-a1ac-579a22ee725b None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:30.197872 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c797a-ec4b-4bab-a1ac-579a22ee725b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:30.198131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71c797a-ec4b-4bab-a1ac-579a22ee725b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:30.205499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c797a-ec4b-4bab-a1ac-579a22ee725b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:30.206965 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71c797a-ec4b-4bab-a1ac-579a22ee725b None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:30.219504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-f71c797a-ec4b-4bab-a1ac-579a22ee725b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:30.219504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:30.219504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:30.219504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:30.219504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:30.219504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:30.219504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:30.219504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:30.219504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:30.219504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:30.219504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:30.219504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:30.219504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:30.219504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:30.219504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:30.220495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:30.220495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:30.220495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:30.220495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:30.220495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:30.220495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:30.220495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:30.220495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:30.220495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:30.220495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:30.220495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:30.220495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:30.220495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:30.220495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:30.220495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:30.220495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:30.220495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:30.221390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:30.221390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:30.221390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:30.221390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:30.221390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:30.221390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:30.221390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:30.221390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:30.221390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 257/519] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:30.221956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c797a-ec4b-4bab-a1ac-579a22ee725b None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:30.243740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41596e-0c9c-4142-a9e1-effe6ba3a4a3 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:30.244456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:30.269248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:30.289426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b6fda-7a67-446e-9a1d-25d31715600a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:30.290056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b6fda-7a67-446e-9a1d-25d31715600a None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:30.290359 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b6fda-7a67-446e-9a1d-25d31715600a None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:30.290661 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b6fda-7a67-446e-9a1d-25d31715600a None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:30.291408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b6fda-7a67-446e-9a1d-25d31715600a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:30.291714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b6fda-7a67-446e-9a1d-25d31715600a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:30.293340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b6fda-7a67-446e-9a1d-25d31715600a None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:30.311489 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 263/520] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:30.312542 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b6fda-7a67-446e-9a1d-25d31715600a None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:30.339164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71c797a-ec4b-4bab-a1ac-579a22ee725b None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:30.379427 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cffeb-c9ee-47b5-9426-c9e00d470587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:30.380227 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cffeb-c9ee-47b5-9426-c9e00d470587 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:30.380500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cffeb-c9ee-47b5-9426-c9e00d470587 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:30.380785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cffeb-c9ee-47b5-9426-c9e00d470587 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:30.452759 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345cffeb-c9ee-47b5-9426-c9e00d470587 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:30.454128 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345cffeb-c9ee-47b5-9426-c9e00d470587 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:30.469142 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b6fda-7a67-446e-9a1d-25d31715600a None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:30.481768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:30.505058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b822d6f-0821-49f2-b8b2-178e221189b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:30.505942 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b822d6f-0821-49f2-b8b2-178e221189b4 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:30.506394 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b822d6f-0821-49f2-b8b2-178e221189b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:30.506662 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b822d6f-0821-49f2-b8b2-178e221189b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:30.520930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b822d6f-0821-49f2-b8b2-178e221189b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:30.522010 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b822d6f-0821-49f2-b8b2-178e221189b4 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:30.538287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-6b822d6f-0821-49f2-b8b2-178e221189b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:30.538287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:30.538287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:30.538287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:30.538287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:30.538287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:30.538287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:30.538287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:30.538287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:30.538287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:30.538287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:30.538287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:30.538287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:30.538287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:30.538287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:30.538287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:30.539421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:30.539421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:30.539421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:30.539421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:30.539421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:30.539421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:30.539421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:30.539421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:30.539421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:30.539421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:30.539421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:30.539421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:30.539421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:30.539421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:30.539421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:30.539421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:30.539421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:30.540608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:30.540608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:30.540608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:30.540608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 264/521] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:30.541319 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b822d6f-0821-49f2-b8b2-178e221189b4 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:30.581555 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 258/522] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:30.582446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cffeb-c9ee-47b5-9426-c9e00d470587 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:30.589634 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:30.668395 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b822d6f-0821-49f2-b8b2-178e221189b4 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:30.687701 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345cffeb-c9ee-47b5-9426-c9e00d470587 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:30.704733 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed47f98-4716-417a-82d1-879eacd1cfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:30.705273 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed47f98-4716-417a-82d1-879eacd1cfa4 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:30.705554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed47f98-4716-417a-82d1-879eacd1cfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:30.705927 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed47f98-4716-417a-82d1-879eacd1cfa4 None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:30.706747 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:30.707777 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed47f98-4716-417a-82d1-879eacd1cfa4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:30.708260 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed47f98-4716-417a-82d1-879eacd1cfa4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:30.710052 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed47f98-4716-417a-82d1-879eacd1cfa4 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:30.724223 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b279831-44b0-41eb-b229-123469fb396c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:30.724943 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b279831-44b0-41eb-b229-123469fb396c None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:30.725295 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b279831-44b0-41eb-b229-123469fb396c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:30.725567 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b279831-44b0-41eb-b229-123469fb396c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:30.728385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 265/523] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:30.729372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed47f98-4716-417a-82d1-879eacd1cfa4 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:30.737717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b279831-44b0-41eb-b229-123469fb396c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:30.739305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b279831-44b0-41eb-b229-123469fb396c None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:30.754047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-2b279831-44b0-41eb-b229-123469fb396c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 19:13:30.754047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:30.754047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:30.754047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:30.754047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:30.754047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:30.754047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:30.754047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:30.754047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:30.754047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:30.754047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:30.754047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:30.754047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:30.754047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:30.754047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:30.754047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:30.755051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:30.756296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:30.756296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:30.756296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:30.756296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:30.756296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:30.756296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:30.756296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:30.756296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:30.756296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 19:13:30.756296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:30.756904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 259/524] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:30.757689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b279831-44b0-41eb-b229-123469fb396c None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:30.832540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:30.846647 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed47f98-4716-417a-82d1-879eacd1cfa4 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:30.868641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b279831-44b0-41eb-b229-123469fb396c None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:30.881788 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b262f0c-745b-4303-8849-13d7bc96dff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:30.882394 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b262f0c-745b-4303-8849-13d7bc96dff6 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:30.882902 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b262f0c-745b-4303-8849-13d7bc96dff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:30.882902 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b262f0c-745b-4303-8849-13d7bc96dff6 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:30.883550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b262f0c-745b-4303-8849-13d7bc96dff6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:30.883847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b262f0c-745b-4303-8849-13d7bc96dff6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:30.885269 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b262f0c-745b-4303-8849-13d7bc96dff6 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:30.897335 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 266/525] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:30.902510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b262f0c-745b-4303-8849-13d7bc96dff6 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:30.907305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636b9f0-d72b-49eb-bc59-b5b812a23ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:30.907956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636b9f0-d72b-49eb-bc59-b5b812a23ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:30.908262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636b9f0-d72b-49eb-bc59-b5b812a23ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:30.908528 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636b9f0-d72b-49eb-bc59-b5b812a23ca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:30.919803 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636b9f0-d72b-49eb-bc59-b5b812a23ca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:30.921278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636b9f0-d72b-49eb-bc59-b5b812a23ca8 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:30.939905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-1636b9f0-d72b-49eb-bc59-b5b812a23ca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:30.939905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:30.939905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:30.939905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:30.939905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:30.939905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:30.939905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:30.939905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:30.939905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:30.939905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:30.939905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:30.939905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:30.939905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:30.939905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:30.939905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:30.939905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:30.941123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:30.941123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:30.941123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:30.941123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:30.941123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:30.941123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:30.941123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:30.941123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:30.941123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:30.941123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:30.941123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:30.941123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:30.941123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:30.941123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:30.941123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:30.941123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:30.941123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:30.941969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:30.941969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:30.941969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:30.941969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 260/526] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:30.941969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636b9f0-d72b-49eb-bc59-b5b812a23ca8 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:30.976237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:31.038102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b262f0c-745b-4303-8849-13d7bc96dff6 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:31.060979 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636b9f0-d72b-49eb-bc59-b5b812a23ca8 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:31.072934 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebe25c-980c-477d-aa28-33dec1848c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:31.073541 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebe25c-980c-477d-aa28-33dec1848c55 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:31.073812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebe25c-980c-477d-aa28-33dec1848c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:31.074113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebe25c-980c-477d-aa28-33dec1848c55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:31.074829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebe25c-980c-477d-aa28-33dec1848c55 None admin] RBAC: Adding query filter params (role.id=43aa4a9830054a09baa1f0aecfd125e2, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=a9daa1a3c7654dcfa5ccb03f467ce7d5) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:31.075127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebe25c-980c-477d-aa28-33dec1848c55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:31.076601 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebe25c-980c-477d-aa28-33dec1848c55 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:31.092053 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 267/527] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:13:30 2022] GET /identity/v3/role_assignments?user.id=a9daa1a3c7654dcfa5ccb03f467ce7d5&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=43aa4a9830054a09baa1f0aecfd125e2 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:31.092053 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebe25c-980c-477d-aa28-33dec1848c55 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:31.109908 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7646fa4-f6dc-4ccb-9443-b8c15fd60c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:31.110703 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:31.111613 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7646fa4-f6dc-4ccb-9443-b8c15fd60c54 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:31.112039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7646fa4-f6dc-4ccb-9443-b8c15fd60c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:31.112558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7646fa4-f6dc-4ccb-9443-b8c15fd60c54 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:31.113435 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7646fa4-f6dc-4ccb-9443-b8c15fd60c54 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:31.113817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7646fa4-f6dc-4ccb-9443-b8c15fd60c54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:31.115427 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7646fa4-f6dc-4ccb-9443-b8c15fd60c54 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:31.136432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 261/528] 10.222.0.17 () {56 vars in 1139 bytes} [Sun Aug 7 19:13:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:31.137451 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7646fa4-f6dc-4ccb-9443-b8c15fd60c54 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:31.205395 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebe25c-980c-477d-aa28-33dec1848c55 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:31.244503 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114feb3-1b37-4954-be38-1fef69f68bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:31.245159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114feb3-1b37-4954-be38-1fef69f68bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:31.245508 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114feb3-1b37-4954-be38-1fef69f68bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:31.245886 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114feb3-1b37-4954-be38-1fef69f68bc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:31.252649 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:31.253766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114feb3-1b37-4954-be38-1fef69f68bc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:31.255529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114feb3-1b37-4954-be38-1fef69f68bc4 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:31.267844 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7646fa4-f6dc-4ccb-9443-b8c15fd60c54 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:31.268514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-5114feb3-1b37-4954-be38-1fef69f68bc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:31.268514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:31.268514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:31.268514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:31.268514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:31.268514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:31.268514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:31.268514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:31.268514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:31.268514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:31.268514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:31.268514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:31.268514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:31.268514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:31.268514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:31.269477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:31.269477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:31.269477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:31.269477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:31.269477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:31.269477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:31.269477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:31.269477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:31.269477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:31.269477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:31.269477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:31.269477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:31.269477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:31.269477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:31.269477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:31.269477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:31.269477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:31.270320 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:31.270320 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:31.270320 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:31.270320 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:31.270320 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:31.270320 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:31.270320 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:31.270320 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:31.270320 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 268/529] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:31.271108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114feb3-1b37-4954-be38-1fef69f68bc4 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:31.313025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02108ca-18af-46ed-a6aa-aad540308f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:31.313652 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02108ca-18af-46ed-a6aa-aad540308f00 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:31.313932 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02108ca-18af-46ed-a6aa-aad540308f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:31.314204 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02108ca-18af-46ed-a6aa-aad540308f00 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:31.314860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02108ca-18af-46ed-a6aa-aad540308f00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:31.315160 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02108ca-18af-46ed-a6aa-aad540308f00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:31.316696 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02108ca-18af-46ed-a6aa-aad540308f00 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:31.338226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 262/530] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:31.339428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02108ca-18af-46ed-a6aa-aad540308f00 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:31.369304 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:31.399666 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114feb3-1b37-4954-be38-1fef69f68bc4 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:31.435812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed75560-c2dd-4020-bb9a-851b51f3d71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:31.436472 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed75560-c2dd-4020-bb9a-851b51f3d71d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:31.436817 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed75560-c2dd-4020-bb9a-851b51f3d71d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:31.437022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed75560-c2dd-4020-bb9a-851b51f3d71d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:31.446704 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02108ca-18af-46ed-a6aa-aad540308f00 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:31.482776 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16addb0a-88ce-49ef-bd5a-187bf0970579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:31.483770 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16addb0a-88ce-49ef-bd5a-187bf0970579 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:31.484287 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16addb0a-88ce-49ef-bd5a-187bf0970579 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:31.484783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16addb0a-88ce-49ef-bd5a-187bf0970579 None admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:31.485690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16addb0a-88ce-49ef-bd5a-187bf0970579 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:31.486210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16addb0a-88ce-49ef-bd5a-187bf0970579 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:31.487836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16addb0a-88ce-49ef-bd5a-187bf0970579 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:31.507638 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 263/531] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:31.508830 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16addb0a-88ce-49ef-bd5a-187bf0970579 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:31.513494 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed75560-c2dd-4020-bb9a-851b51f3d71d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:31.514995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed75560-c2dd-4020-bb9a-851b51f3d71d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:31.544791 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:31.598742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:31.616673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16addb0a-88ce-49ef-bd5a-187bf0970579 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:31.637751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 269/532] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:31.637751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed75560-c2dd-4020-bb9a-851b51f3d71d None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:31.659059 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28feec-ffca-4a8e-abd5-a6f5652f4d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:31.660097 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28feec-ffca-4a8e-abd5-a6f5652f4d0c None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:31.660577 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28feec-ffca-4a8e-abd5-a6f5652f4d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:31.661071 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28feec-ffca-4a8e-abd5-a6f5652f4d0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:31.669689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a28feec-ffca-4a8e-abd5-a6f5652f4d0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:31.671474 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a28feec-ffca-4a8e-abd5-a6f5652f4d0c None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:31.684042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-4a28feec-ffca-4a8e-abd5-a6f5652f4d0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:31.684042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:31.684042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:31.684042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:31.684042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:31.684042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:31.684042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:31.684042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:31.684042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:31.684042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:31.684042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:31.684042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:31.684042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:31.684042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:31.684042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:31.685024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:31.685024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:31.685024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:31.685024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:31.685024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:31.685024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:31.685024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:31.685024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:31.685024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:31.685024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:31.685024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:31.685024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:31.685024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:31.685024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:31.685024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:31.685024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:31.685024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:31.685881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:31.685881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:31.685881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:31.685881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:31.685881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:31.685881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:31.685881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:31.685881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:31.687980 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 264/533] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:31.693089 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28feec-ffca-4a8e-abd5-a6f5652f4d0c None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:31.740115 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed75560-c2dd-4020-bb9a-851b51f3d71d None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:31.764499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:31.775161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38bf26-cbc4-4526-b6e3-33f0fc942235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:31.775829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38bf26-cbc4-4526-b6e3-33f0fc942235 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:31.776073 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38bf26-cbc4-4526-b6e3-33f0fc942235 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:31.776351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c38bf26-cbc4-4526-b6e3-33f0fc942235 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:31.777674 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38bf26-cbc4-4526-b6e3-33f0fc942235 None admin] RBAC: Adding query filter params (role.id=43aa4a9830054a09baa1f0aecfd125e2, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=f83e5f0b1d5e4f57bbf283d420f080ec) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:31.778267 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38bf26-cbc4-4526-b6e3-33f0fc942235 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:31.780124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c38bf26-cbc4-4526-b6e3-33f0fc942235 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:31.785821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 270/534] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:13:31 2022] GET /identity/v3/role_assignments?user.id=f83e5f0b1d5e4f57bbf283d420f080ec&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=43aa4a9830054a09baa1f0aecfd125e2 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:31.786884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38bf26-cbc4-4526-b6e3-33f0fc942235 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:31.799421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28feec-ffca-4a8e-abd5-a6f5652f4d0c None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:31.837750 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609399d-fce5-4481-9cd4-4ff73fb83637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:31.838828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609399d-fce5-4481-9cd4-4ff73fb83637 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:31.839332 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609399d-fce5-4481-9cd4-4ff73fb83637 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:31.839896 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609399d-fce5-4481-9cd4-4ff73fb83637 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:31.852340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609399d-fce5-4481-9cd4-4ff73fb83637 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:31.854247 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609399d-fce5-4481-9cd4-4ff73fb83637 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:31.871626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-c609399d-fce5-4481-9cd4-4ff73fb83637 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:31.871626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:31.871626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:31.871626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:31.871626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:31.871626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:31.871626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:31.871626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:31.871626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:31.871626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:31.871626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:31.871626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:31.871626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:31.871626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:31.871626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:31.871626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:31.873074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:31.873074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:31.873074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:31.873074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:31.873074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:31.873074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:31.873074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:31.873074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:31.873074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:31.873074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:31.873074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:31.873074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:31.873074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:31.873074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:31.873074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:31.873074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:31.873074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:31.873920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:31.873920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:31.873920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:31.875210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:31.877548 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 265/535] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:31.884477 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609399d-fce5-4481-9cd4-4ff73fb83637 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:31.992261 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c38bf26-cbc4-4526-b6e3-33f0fc942235 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:32.005227 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609399d-fce5-4481-9cd4-4ff73fb83637 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:32.041061 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:32.059884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b465d-4c13-4454-97d9-f9da3437718c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:32.061025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b465d-4c13-4454-97d9-f9da3437718c None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:32.061609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b465d-4c13-4454-97d9-f9da3437718c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:32.062187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b465d-4c13-4454-97d9-f9da3437718c None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:32.063065 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b465d-4c13-4454-97d9-f9da3437718c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:32.063574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b465d-4c13-4454-97d9-f9da3437718c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:32.065462 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b465d-4c13-4454-97d9-f9da3437718c None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:32.072474 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2aab5a-465b-4ba3-9a8a-a47cf054016b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:32.073172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2aab5a-465b-4ba3-9a8a-a47cf054016b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:32.073444 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2aab5a-465b-4ba3-9a8a-a47cf054016b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:32.073707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2aab5a-465b-4ba3-9a8a-a47cf054016b None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:32.074502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2aab5a-465b-4ba3-9a8a-a47cf054016b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:32.074803 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2aab5a-465b-4ba3-9a8a-a47cf054016b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:32.076342 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2aab5a-465b-4ba3-9a8a-a47cf054016b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:32.082442 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 266/536] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:32.093144 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b465d-4c13-4454-97d9-f9da3437718c None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:32.108891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 271/537] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:32.117512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2aab5a-465b-4ba3-9a8a-a47cf054016b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:32.230747 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b465d-4c13-4454-97d9-f9da3437718c None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:32.243267 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:32.253686 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2aab5a-465b-4ba3-9a8a-a47cf054016b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:32.281844 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc8641-9960-48c9-89ae-f127136f7beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:32.282896 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc8641-9960-48c9-89ae-f127136f7beb None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:32.283374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc8641-9960-48c9-89ae-f127136f7beb None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:32.283862 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc8641-9960-48c9-89ae-f127136f7beb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:32.294092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc8641-9960-48c9-89ae-f127136f7beb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:32.296511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc8641-9960-48c9-89ae-f127136f7beb None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:32.299427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6153c5e0-6025-43fb-848f-9d0e2c57c236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:32.300487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153c5e0-6025-43fb-848f-9d0e2c57c236 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:32.300763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153c5e0-6025-43fb-848f-9d0e2c57c236 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:32.301028 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6153c5e0-6025-43fb-848f-9d0e2c57c236 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:32.301754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153c5e0-6025-43fb-848f-9d0e2c57c236 None admin] RBAC: Adding query filter params (role.id=43aa4a9830054a09baa1f0aecfd125e2, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=a4946aec299f45bd8402cff3159d04d7) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:32.302053 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153c5e0-6025-43fb-848f-9d0e2c57c236 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:32.303536 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6153c5e0-6025-43fb-848f-9d0e2c57c236 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:32.316038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 272/538] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:13:32 2022] GET /identity/v3/role_assignments?user.id=a4946aec299f45bd8402cff3159d04d7&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=43aa4a9830054a09baa1f0aecfd125e2 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:32.316038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c0665-c17d-4262-ac2d-bdb2865baada None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:32.316038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c0665-c17d-4262-ac2d-bdb2865baada None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:32.316038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c0665-c17d-4262-ac2d-bdb2865baada None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:32.316038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 273/539] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:32.317712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-3bcc8641-9960-48c9-89ae-f127136f7beb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:32.317712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:32.317712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:32.317712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:32.317712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:32.317712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:32.317712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:32.317712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:32.317712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:32.317712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:32.317712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:32.317712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:32.317712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:32.317712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:32.317712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:32.318511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:32.318511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:32.318511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:32.318511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:32.318511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:32.318511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:32.318511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:32.318511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:32.318511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:32.318511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:32.318511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:32.318511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:32.318511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:32.318511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:32.318511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:32.318511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:32.318511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:32.319374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:32.319374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:32.319374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:32.319374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:32.319374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:32.319374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:32.319374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:32.319374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:32.319374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 267/540] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:32.325650 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74c2b6-de65-4673-af05-2482433dc945 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:32.325933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74c2b6-de65-4673-af05-2482433dc945 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:32.326199 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74c2b6-de65-4673-af05-2482433dc945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:32.329267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc8641-9960-48c9-89ae-f127136f7beb None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:32.364402 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-7c74c2b6-de65-4673-af05-2482433dc945 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:32.365641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-7c74c2b6-de65-4673-af05-2482433dc945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:32.424895 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:32.506925 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 274/541] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:32.529399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74c2b6-de65-4673-af05-2482433dc945 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:32.533942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc8641-9960-48c9-89ae-f127136f7beb None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:32.576223 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe2e8a-eee5-4ea9-bafb-8abfbc9fb758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:32.576763 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe2e8a-eee5-4ea9-bafb-8abfbc9fb758 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:32.577037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe2e8a-eee5-4ea9-bafb-8abfbc9fb758 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:32.577308 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe2e8a-eee5-4ea9-bafb-8abfbc9fb758 None admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:32.578011 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe2e8a-eee5-4ea9-bafb-8abfbc9fb758 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:32.578315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe2e8a-eee5-4ea9-bafb-8abfbc9fb758 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:32.579762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe2e8a-eee5-4ea9-bafb-8abfbc9fb758 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:32.593362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 268/542] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:32.599360 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe2e8a-eee5-4ea9-bafb-8abfbc9fb758 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:32.677593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:32.686567 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74c2b6-de65-4673-af05-2482433dc945 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:32.726223 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11688c-0995-464e-8b87-d63ae9502a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:32.727240 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11688c-0995-464e-8b87-d63ae9502a59 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:32.727735 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11688c-0995-464e-8b87-d63ae9502a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:32.728375 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11688c-0995-464e-8b87-d63ae9502a59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:32.741328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11688c-0995-464e-8b87-d63ae9502a59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:32.747408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-7e11688c-0995-464e-8b87-d63ae9502a59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:32.747408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:32.747408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:32.747408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:32.747408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:32.747408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:32.747408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:32.747408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:32.747408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:32.747408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:32.747408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:32.747408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:32.747408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:32.747408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:32.747408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:32.747408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:32.748379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:32.748379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:32.748379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:32.748379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:32.748379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:32.748379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:32.748379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:32.748379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:32.748379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:32.748379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:32.748379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:32.748379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:32.748379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:32.748379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:32.748379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:32.748379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:32.749345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:32.749345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:32.749345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:32.749345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:32.749345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:32.749345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:32.749345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:32.749345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:32.749345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:32.749345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:32.749345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:32.749345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:32.751849 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 275/543] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:32.759769 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11688c-0995-464e-8b87-d63ae9502a59 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:32.804223 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:32.809133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe2e8a-eee5-4ea9-bafb-8abfbc9fb758 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:32.877349 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b73fa8-161c-4672-b706-eb5148e0bfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:32.878712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b73fa8-161c-4672-b706-eb5148e0bfc8 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:32.879162 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b73fa8-161c-4672-b706-eb5148e0bfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:32.879595 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b73fa8-161c-4672-b706-eb5148e0bfc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:32.880792 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b73fa8-161c-4672-b706-eb5148e0bfc8 None admin] RBAC: Adding query filter params (role.id=43aa4a9830054a09baa1f0aecfd125e2, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=08229e9400b14e8a9cc8e8c32dc48285) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:32.882024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b73fa8-161c-4672-b706-eb5148e0bfc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:32.884470 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b73fa8-161c-4672-b706-eb5148e0bfc8 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:32.901072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 269/544] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:13:32 2022] GET /identity/v3/role_assignments?user.id=08229e9400b14e8a9cc8e8c32dc48285&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=43aa4a9830054a09baa1f0aecfd125e2 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:32.960030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11688c-0995-464e-8b87-d63ae9502a59 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:33.035328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c792c-1c20-4584-98b9-4f4628899d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:33.036039 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c792c-1c20-4584-98b9-4f4628899d51 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:33.036250 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c792c-1c20-4584-98b9-4f4628899d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:33.036519 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c792c-1c20-4584-98b9-4f4628899d51 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:33.037210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c792c-1c20-4584-98b9-4f4628899d51 None admin] RBAC: Adding query filter params (name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:33.037509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c792c-1c20-4584-98b9-4f4628899d51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:33.038949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c792c-1c20-4584-98b9-4f4628899d51 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:33.051772 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 276/545] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:13:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:33.058021 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b73fa8-161c-4672-b706-eb5148e0bfc8 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:33.194509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b73fa8-161c-4672-b706-eb5148e0bfc8 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:33.232445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23ac85-0a43-43c7-a168-e07b285e8953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:33.233588 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23ac85-0a43-43c7-a168-e07b285e8953 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:33.234033 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23ac85-0a43-43c7-a168-e07b285e8953 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:33.234465 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23ac85-0a43-43c7-a168-e07b285e8953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:33.246205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23ac85-0a43-43c7-a168-e07b285e8953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:33.247959 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23ac85-0a43-43c7-a168-e07b285e8953 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:33.263346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-9f23ac85-0a43-43c7-a168-e07b285e8953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:33.263346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:33.263346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:33.263346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:33.263346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:33.263346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:33.263346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:33.263346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:33.263346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:33.263346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:33.263346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:33.263346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:33.263346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:33.263346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:33.263346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:33.263346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:33.264320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:33.264320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:33.264320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:33.264320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:33.264320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:33.264320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:33.264320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:33.264320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:33.264320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:33.264320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:33.264320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:33.264320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:33.264320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:33.264320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:33.264320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:33.264320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:33.264320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:33.265164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:33.265164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:33.265164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:33.267337 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 270/546] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:33.272918 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c792c-1c20-4584-98b9-4f4628899d51 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:33.296425 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:33.377003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd010c-8a2a-4bff-b688-04be275128a8 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:33.377628 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd010c-8a2a-4bff-b688-04be275128a8 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:33.378088 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd010c-8a2a-4bff-b688-04be275128a8 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:33.381132 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 271/547] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:33.385802 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49257e88-2fc9-48a2-a28e-02e49dddb2ff None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:33.386400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49257e88-2fc9-48a2-a28e-02e49dddb2ff None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:33.386864 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49257e88-2fc9-48a2-a28e-02e49dddb2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:33.388074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c792c-1c20-4584-98b9-4f4628899d51 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:33.424049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a924ea9-6a32-4ef9-a1d6-6595f931f841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:33.424703 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a924ea9-6a32-4ef9-a1d6-6595f931f841 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:33.424983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a924ea9-6a32-4ef9-a1d6-6595f931f841 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:33.425255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a924ea9-6a32-4ef9-a1d6-6595f931f841 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:33.425912 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a924ea9-6a32-4ef9-a1d6-6595f931f841 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:33.426216 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a924ea9-6a32-4ef9-a1d6-6595f931f841 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:33.427557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-49257e88-2fc9-48a2-a28e-02e49dddb2ff None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:33.427677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a924ea9-6a32-4ef9-a1d6-6595f931f841 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:33.434940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-49257e88-2fc9-48a2-a28e-02e49dddb2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:33.444038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 277/548] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:33.445841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a924ea9-6a32-4ef9-a1d6-6595f931f841 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:33.457911 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:33.544235 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:33.568083 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a924ea9-6a32-4ef9-a1d6-6595f931f841 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:33.585724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 272/549] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:13:33.607947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49257e88-2fc9-48a2-a28e-02e49dddb2ff None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:33.612418 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11629f77-d74e-43f5-ac79-ef6d92f840a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:33.612418 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11629f77-d74e-43f5-ac79-ef6d92f840a3 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:33.612668 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11629f77-d74e-43f5-ac79-ef6d92f840a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:33.612851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11629f77-d74e-43f5-ac79-ef6d92f840a3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:33.629585 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11629f77-d74e-43f5-ac79-ef6d92f840a3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:33.631100 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11629f77-d74e-43f5-ac79-ef6d92f840a3 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:33.646509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-11629f77-d74e-43f5-ac79-ef6d92f840a3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 19:13:33.646509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:33.646509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:33.646509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:33.646509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:33.646509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:33.646509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:33.646509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:33.646509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:33.646509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:33.646509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:33.646509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:33.646509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:33.646509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:33.646509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:33.646509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:33.647971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:33.649057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:33.649057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:33.649057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:33.649057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:33.649057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:33.649057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:33.649057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:33.649057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:33.649057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 19:13:33.649057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:33.655955 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 278/550] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:13:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:33.661793 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11629f77-d74e-43f5-ac79-ef6d92f840a3 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:33.766511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49257e88-2fc9-48a2-a28e-02e49dddb2ff None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:33.832009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba079397-4b22-4ba6-aa24-028b39edbde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:33.832673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba079397-4b22-4ba6-aa24-028b39edbde0 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:33.832948 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba079397-4b22-4ba6-aa24-028b39edbde0 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:33.833227 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba079397-4b22-4ba6-aa24-028b39edbde0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:33.850658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11629f77-d74e-43f5-ac79-ef6d92f840a3 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:33.856023 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba079397-4b22-4ba6-aa24-028b39edbde0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:33.862074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-ba079397-4b22-4ba6-aa24-028b39edbde0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:33.862074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:33.862074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:33.862074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:33.862074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:33.862074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:33.862074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:33.862074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:33.862074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:33.862074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:33.862074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:33.862074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:33.862074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:33.862074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:33.862074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:33.862074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:33.863000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:33.863000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:33.863000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:33.863000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:33.863000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:33.863000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:33.863000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:33.863000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:33.863000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:33.863000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:33.863000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:33.863000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:33.863000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:33.863000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:33.863000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:33.863000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:33.864340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:33.864340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:33.864340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:33.864340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:33.864340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:33.864340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:33.864340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:33.864340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:33.864340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:33.864340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:33.864340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:33.864340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:33.864340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 273/551] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:33.872718 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba079397-4b22-4ba6-aa24-028b39edbde0 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:33.892913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:33.898323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df228558-f307-496a-94a6-1cb2d6e7de84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:33.898713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df228558-f307-496a-94a6-1cb2d6e7de84 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:33.898997 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df228558-f307-496a-94a6-1cb2d6e7de84 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:33.899269 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df228558-f307-496a-94a6-1cb2d6e7de84 None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:33.900109 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df228558-f307-496a-94a6-1cb2d6e7de84 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:33.900416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df228558-f307-496a-94a6-1cb2d6e7de84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:33.901869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df228558-f307-496a-94a6-1cb2d6e7de84 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:33.904924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:33.922428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 279/552] 10.222.0.17 () {56 vars in 1135 bytes} [Sun Aug 7 19:13:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:33.931497 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df228558-f307-496a-94a6-1cb2d6e7de84 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:34.052042 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df228558-f307-496a-94a6-1cb2d6e7de84 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:34.095481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a11470-c548-4d58-b20f-e482e462f722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:34.096206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a11470-c548-4d58-b20f-e482e462f722 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:34.096487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a11470-c548-4d58-b20f-e482e462f722 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:34.096758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a11470-c548-4d58-b20f-e482e462f722 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:34.172134 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba079397-4b22-4ba6-aa24-028b39edbde0 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:34.218122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:34.227515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a11470-c548-4d58-b20f-e482e462f722 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:34.232015 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a11470-c548-4d58-b20f-e482e462f722 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:34.261504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4db03c-bf60-4bc5-9573-c4a025632f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:34.262098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4db03c-bf60-4bc5-9573-c4a025632f00 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:34.262380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4db03c-bf60-4bc5-9573-c4a025632f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:34.262654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4db03c-bf60-4bc5-9573-c4a025632f00 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:34.263334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4db03c-bf60-4bc5-9573-c4a025632f00 None admin] RBAC: Adding query filter params (name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:34.263639 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4db03c-bf60-4bc5-9573-c4a025632f00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:34.265078 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4db03c-bf60-4bc5-9573-c4a025632f00 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:34.286796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 274/553] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:13:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:34.293177 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4db03c-bf60-4bc5-9573-c4a025632f00 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:34.387102 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:34.412792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 280/554] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:34.419794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a11470-c548-4d58-b20f-e482e462f722 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:34.453984 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4db03c-bf60-4bc5-9573-c4a025632f00 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:34.505364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee68c87-6ae9-42de-8ecb-99e6518870da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:34.506014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee68c87-6ae9-42de-8ecb-99e6518870da None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:34.506295 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee68c87-6ae9-42de-8ecb-99e6518870da None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:34.506568 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee68c87-6ae9-42de-8ecb-99e6518870da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:34.521781 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee68c87-6ae9-42de-8ecb-99e6518870da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:34.523509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee68c87-6ae9-42de-8ecb-99e6518870da None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:34.543688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-aee68c87-6ae9-42de-8ecb-99e6518870da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:34.543688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:34.543688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:34.543688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:34.543688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:34.543688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:34.543688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:34.543688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:34.543688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:34.543688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:34.543688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:34.543688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:34.543688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:34.543688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:34.543688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:34.543688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:34.544709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:34.544709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:34.544709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:34.544709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:34.544709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:34.544709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:34.544709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:34.544709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:34.544709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:34.544709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:34.544709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:34.544709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:34.544709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:34.544709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:34.544709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:34.544709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:34.544709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:34.546043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:34.546043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:34.546043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:34.546043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 275/555] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:34.547302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b9e1a-ce05-47c3-b9ea-7ca66a47d889 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:34.547413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b9e1a-ce05-47c3-b9ea-7ca66a47d889 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:34.547684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b9e1a-ce05-47c3-b9ea-7ca66a47d889 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:34.549271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 276/556] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:34.552439 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b9e1a-ce05-47c3-b9ea-7ca66a47d889 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:34.563672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:34.572745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a11470-c548-4d58-b20f-e482e462f722 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:34.619313 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e98112-3282-424a-9d55-b51b8a7f752d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:34.620328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e98112-3282-424a-9d55-b51b8a7f752d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:34.620990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e98112-3282-424a-9d55-b51b8a7f752d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:34.621337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e98112-3282-424a-9d55-b51b8a7f752d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:34.635516 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e98112-3282-424a-9d55-b51b8a7f752d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:34.638048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e98112-3282-424a-9d55-b51b8a7f752d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:34.658341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-95e98112-3282-424a-9d55-b51b8a7f752d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:34.658341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:34.658341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:34.658341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:34.658341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:34.658341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:34.658341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:34.658341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:34.658341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:34.658341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:34.658341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:34.658341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:34.658341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:34.658341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:34.658341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:34.658341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:34.659374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:34.659374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:34.659374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:34.659374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:34.659374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:34.659374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:34.659374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:34.659374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:34.659374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:34.659374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:34.659374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:34.659374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:34.659374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:34.659374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:34.659374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:34.659374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:34.659374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:34.661528 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:34.661528 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:34.661528 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:34.661528 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 281/557] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:34.661528 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c6ff1-8255-48e6-a2e4-8726fcb74eb3 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:34.662012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c6ff1-8255-48e6-a2e4-8726fcb74eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:34.662012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c6ff1-8255-48e6-a2e4-8726fcb74eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:34.681020 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-cd2c6ff1-8255-48e6-a2e4-8726fcb74eb3 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:34.682228 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-cd2c6ff1-8255-48e6-a2e4-8726fcb74eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:34.685935 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b9e1a-ce05-47c3-b9ea-7ca66a47d889 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:34.708150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:34.724682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d19b6-ff10-4a28-841f-2272d63af720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:34.725309 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d19b6-ff10-4a28-841f-2272d63af720 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:34.725593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d19b6-ff10-4a28-841f-2272d63af720 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:34.725827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d19b6-ff10-4a28-841f-2272d63af720 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:34.726487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d19b6-ff10-4a28-841f-2272d63af720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:34.726787 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d19b6-ff10-4a28-841f-2272d63af720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:34.728200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d19b6-ff10-4a28-841f-2272d63af720 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:34.745459 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 277/558] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:34.745459 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a157fff-bef9-43d0-942c-0676ab047230 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:34.745459 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a157fff-bef9-43d0-942c-0676ab047230 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:34.745459 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a157fff-bef9-43d0-942c-0676ab047230 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:34.745459 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 278/559] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:34.745459 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a157fff-bef9-43d0-942c-0676ab047230 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:34.764382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 282/560] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:34.765111 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c6ff1-8255-48e6-a2e4-8726fcb74eb3 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:34.845213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:34.868937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a157fff-bef9-43d0-942c-0676ab047230 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:34.870122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c6ff1-8255-48e6-a2e4-8726fcb74eb3 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:34.906686 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd967b3-546e-4012-995b-c5e05307c360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:34.907278 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd967b3-546e-4012-995b-c5e05307c360 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:34.907626 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd967b3-546e-4012-995b-c5e05307c360 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:34.907821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd967b3-546e-4012-995b-c5e05307c360 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:34.910173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284b760-cffa-41ad-b540-42e16d2e099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:34.910808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284b760-cffa-41ad-b540-42e16d2e099b None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:34.911065 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284b760-cffa-41ad-b540-42e16d2e099b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:34.911334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284b760-cffa-41ad-b540-42e16d2e099b None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:34.911999 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c284b760-cffa-41ad-b540-42e16d2e099b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:34.912327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c284b760-cffa-41ad-b540-42e16d2e099b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:34.913701 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c284b760-cffa-41ad-b540-42e16d2e099b None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:34.916483 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd967b3-546e-4012-995b-c5e05307c360 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:34.918018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd967b3-546e-4012-995b-c5e05307c360 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:34.930537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 279/561] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:34.930537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbce2f1-ec95-4531-99c9-eaab28753db0 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:34.930537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbce2f1-ec95-4531-99c9-eaab28753db0 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:34.930537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbce2f1-ec95-4531-99c9-eaab28753db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:34.930952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-4bd967b3-546e-4012-995b-c5e05307c360 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 07 19:13:34.930952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:34.930952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:34.930952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:34.930952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:34.930952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:34.930952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:34.930952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:34.930952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:34.930952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:34.930952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:34.930952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:34.930952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:34.930952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:34.930952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:34.930952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:34.931809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:34.933156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:34.933156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:34.933156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:34.933156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:34.933156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:34.933156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:34.933156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:34.933156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:34.933156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 07 19:13:34.933156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:34.933156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 283/562] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:34.933156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd967b3-546e-4012-995b-c5e05307c360 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:34.954935 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-1bbce2f1-ec95-4531-99c9-eaab28753db0 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:34.956171 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1bbce2f1-ec95-4531-99c9-eaab28753db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:34.992478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:35.041693 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 280/563] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:35.052248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbce2f1-ec95-4531-99c9-eaab28753db0 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:35.057996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd967b3-546e-4012-995b-c5e05307c360 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:35.095636 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b3eb1-0348-4338-95ab-b60d41960fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:35.096526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b3eb1-0348-4338-95ab-b60d41960fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:35.096924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b3eb1-0348-4338-95ab-b60d41960fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:35.097197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b3eb1-0348-4338-95ab-b60d41960fc4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:35.100727 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:35.111741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b3eb1-0348-4338-95ab-b60d41960fc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:35.116933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-dc8b3eb1-0348-4338-95ab-b60d41960fc4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:35.116933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:35.116933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:35.116933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:35.116933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:35.116933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:35.116933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:35.116933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:35.116933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:35.116933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:35.116933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:35.116933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:35.116933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:35.116933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:35.116933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:35.116933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:35.117897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:35.117897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:35.117897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:35.117897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:35.117897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:35.117897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:35.117897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:35.117897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:35.117897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:35.117897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:35.117897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:35.117897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:35.117897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:35.117897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:35.117897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:35.117897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:35.118741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:35.118741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:35.118741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:35.118741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:35.118741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:35.118741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:35.118741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:35.118741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:35.118741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:35.118741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:35.118741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:35.118741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:35.118741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 284/564] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:35.119438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b3eb1-0348-4338-95ab-b60d41960fc4 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:35.153006 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbce2f1-ec95-4531-99c9-eaab28753db0 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:35.188790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc91f5-30bb-4e4a-b190-86dff0502b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:35.189431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc91f5-30bb-4e4a-b190-86dff0502b2b None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:35.189709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc91f5-30bb-4e4a-b190-86dff0502b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:35.189979 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc91f5-30bb-4e4a-b190-86dff0502b2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:35.197057 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadc91f5-30bb-4e4a-b190-86dff0502b2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:35.198514 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadc91f5-30bb-4e4a-b190-86dff0502b2b None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:35.210193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-eadc91f5-30bb-4e4a-b190-86dff0502b2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:35.210193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:35.210193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:35.210193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:35.210193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:35.210193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:35.210193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:35.210193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:35.210193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:35.210193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:35.210193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:35.210193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:35.210193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:35.210193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:35.210193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:35.212320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:35.212320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:35.212320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:35.212320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:35.212320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:35.212320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:35.212320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:35.212320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:35.212320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:35.212320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:35.212320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:35.212320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:35.212320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:35.212320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:35.212320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:35.212320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:35.212320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:35.214413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:35.214413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:35.214413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:35.214413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:35.214413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:35.214413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:35.214413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:35.214413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:35.214413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 281/565] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:35.214413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc91f5-30bb-4e4a-b190-86dff0502b2b None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:35.233275 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b3eb1-0348-4338-95ab-b60d41960fc4 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:35.242877 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:35.267918 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8708d-d398-40bf-aaaa-7dffb3084c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:35.268397 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8708d-d398-40bf-aaaa-7dffb3084c39 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:35.268665 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8708d-d398-40bf-aaaa-7dffb3084c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:35.268977 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8708d-d398-40bf-aaaa-7dffb3084c39 None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:35.269697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8708d-d398-40bf-aaaa-7dffb3084c39 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:35.270002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8708d-d398-40bf-aaaa-7dffb3084c39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:35.271464 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b8708d-d398-40bf-aaaa-7dffb3084c39 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:35.289595 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 285/566] 10.222.0.17 () {56 vars in 1137 bytes} [Sun Aug 7 19:13:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:35.290232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8708d-d398-40bf-aaaa-7dffb3084c39 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:35.312835 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc91f5-30bb-4e4a-b190-86dff0502b2b None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:35.345419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:35.348189 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc50f26e-f668-4a4a-a5c7-f2fca55bf589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:35.348890 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50f26e-f668-4a4a-a5c7-f2fca55bf589 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:35.349177 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50f26e-f668-4a4a-a5c7-f2fca55bf589 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:35.349679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc50f26e-f668-4a4a-a5c7-f2fca55bf589 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:35.359673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc50f26e-f668-4a4a-a5c7-f2fca55bf589 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:35.364808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-fc50f26e-f668-4a4a-a5c7-f2fca55bf589 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:35.364808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:35.364808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:35.364808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:35.364808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:35.364808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:35.364808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:35.364808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:35.364808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:35.364808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:35.364808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:35.364808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:35.364808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:35.364808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:35.364808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:35.364808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:35.365925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:35.365925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:35.365925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:35.365925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:35.365925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:35.365925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:35.365925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:35.365925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:35.365925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:35.365925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:35.365925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:35.365925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:35.365925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:35.365925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:35.365925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:35.365925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:35.367103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:35.367103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:35.367103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:35.367103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:35.367103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:35.367103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:35.367103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:35.367103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:35.367103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:35.367103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:35.367103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:35.367103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:35.367103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 282/567] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:35.367951 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c187c6-2229-4a61-bc67-58d3f97b7732 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:35.367951 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c187c6-2229-4a61-bc67-58d3f97b7732 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:35.367951 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c187c6-2229-4a61-bc67-58d3f97b7732 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:35.369163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 283/568] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:35.369857 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c187c6-2229-4a61-bc67-58d3f97b7732 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:35.395537 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8708d-d398-40bf-aaaa-7dffb3084c39 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:35.432980 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4214a-1a4f-4b0e-a326-849d80b20b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:35.433639 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4214a-1a4f-4b0e-a326-849d80b20b24 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:35.433914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4214a-1a4f-4b0e-a326-849d80b20b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:35.434184 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4214a-1a4f-4b0e-a326-849d80b20b24 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:35.434847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4214a-1a4f-4b0e-a326-849d80b20b24 None admin] RBAC: Adding query filter params (name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:35.435147 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4214a-1a4f-4b0e-a326-849d80b20b24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:35.436570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4214a-1a4f-4b0e-a326-849d80b20b24 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:35.444763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 286/569] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:13:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:35.445517 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4214a-1a4f-4b0e-a326-849d80b20b24 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:35.462941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:35.474960 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c187c6-2229-4a61-bc67-58d3f97b7732 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:35.510100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6962ce6-7456-4442-8355-60f3467d0524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:35.510721 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6962ce6-7456-4442-8355-60f3467d0524 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:35.511000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6962ce6-7456-4442-8355-60f3467d0524 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:35.511271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6962ce6-7456-4442-8355-60f3467d0524 None admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:35.512047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6962ce6-7456-4442-8355-60f3467d0524 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:35.512362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6962ce6-7456-4442-8355-60f3467d0524 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:35.513796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6962ce6-7456-4442-8355-60f3467d0524 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:35.530113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 284/570] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:35.530113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6962ce6-7456-4442-8355-60f3467d0524 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:35.554678 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4214a-1a4f-4b0e-a326-849d80b20b24 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:35.571780 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:35.589408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c80ac85-c61f-4e84-a8b1-c00902621c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:35.590107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80ac85-c61f-4e84-a8b1-c00902621c8c None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:35.590383 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80ac85-c61f-4e84-a8b1-c00902621c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:35.590657 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c80ac85-c61f-4e84-a8b1-c00902621c8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:35.601543 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80ac85-c61f-4e84-a8b1-c00902621c8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:35.603012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c80ac85-c61f-4e84-a8b1-c00902621c8c None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:35.617043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-8c80ac85-c61f-4e84-a8b1-c00902621c8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:35.617043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:35.617043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:35.617043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:35.617043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:35.617043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:35.617043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:35.617043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:35.617043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:35.617043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:35.617043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:35.617043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:35.617043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:35.617043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:35.617043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:35.617043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:35.618021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:35.618021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:35.618021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:35.618021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:35.618021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:35.618021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:35.618021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:35.618021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:35.618021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:35.618021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:35.618021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:35.618021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:35.618021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:35.618021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:35.618021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:35.618021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:35.618021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:35.618911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:35.618911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:35.618911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:35.618911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 287/571] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:35.619324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487fdc3-a08c-4a6c-b461-d39b857cf52a None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:35.619615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487fdc3-a08c-4a6c-b461-d39b857cf52a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:35.619849 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487fdc3-a08c-4a6c-b461-d39b857cf52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:35.634293 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6962ce6-7456-4442-8355-60f3467d0524 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:35.638030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-e487fdc3-a08c-4a6c-b461-d39b857cf52a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:35.639243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e487fdc3-a08c-4a6c-b461-d39b857cf52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:35.667478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807ab1c-2fa0-47a4-b74f-968deb4c5952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:35.668279 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807ab1c-2fa0-47a4-b74f-968deb4c5952 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:35.668564 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807ab1c-2fa0-47a4-b74f-968deb4c5952 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:35.668838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807ab1c-2fa0-47a4-b74f-968deb4c5952 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:35.669510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807ab1c-2fa0-47a4-b74f-968deb4c5952 None admin] RBAC: Adding query filter params (name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:35.669818 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807ab1c-2fa0-47a4-b74f-968deb4c5952 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:35.671195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807ab1c-2fa0-47a4-b74f-968deb4c5952 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:35.679941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 285/572] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:13:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:35.680806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807ab1c-2fa0-47a4-b74f-968deb4c5952 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:35.695733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:35.712139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 288/573] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:35.713014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487fdc3-a08c-4a6c-b461-d39b857cf52a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:35.786060 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807ab1c-2fa0-47a4-b74f-968deb4c5952 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:35.812664 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:35.816968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487fdc3-a08c-4a6c-b461-d39b857cf52a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:35.820597 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1ea00-b8ce-4376-ad90-0f621c8eb250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:35.821239 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1ea00-b8ce-4376-ad90-0f621c8eb250 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:35.821516 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1ea00-b8ce-4376-ad90-0f621c8eb250 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:35.821786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1ea00-b8ce-4376-ad90-0f621c8eb250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:35.833492 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1ea00-b8ce-4376-ad90-0f621c8eb250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:35.834929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d1ea00-b8ce-4376-ad90-0f621c8eb250 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:35.848982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-29d1ea00-b8ce-4376-ad90-0f621c8eb250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:35.848982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:35.848982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:35.848982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:35.848982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:35.848982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:35.848982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:35.848982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:35.848982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:35.848982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:35.848982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:35.848982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:35.848982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:35.848982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:35.848982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:35.848982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:35.850121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:35.850121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:35.850121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:35.850121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:35.850121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:35.850121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:35.850121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:35.850121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:35.850121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:35.850121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:35.850121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:35.850121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:35.850121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:35.850121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:35.850121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:35.850121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:35.850121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:35.851170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:35.851170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:35.851170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:35.851170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 286/574] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:35.851170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1ea00-b8ce-4376-ad90-0f621c8eb250 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:35.857734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682367a-a38a-4e40-85cc-55f1fd970802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:35.857734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682367a-a38a-4e40-85cc-55f1fd970802 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:35.857734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682367a-a38a-4e40-85cc-55f1fd970802 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:35.857734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682367a-a38a-4e40-85cc-55f1fd970802 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:35.858165 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682367a-a38a-4e40-85cc-55f1fd970802 None admin] RBAC: Adding query filter params (role.id=43aa4a9830054a09baa1f0aecfd125e2, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=db51971158a946c98a8ed716ecc5d72b) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:35.858165 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682367a-a38a-4e40-85cc-55f1fd970802 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:35.859310 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682367a-a38a-4e40-85cc-55f1fd970802 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:35.865402 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 289/575] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:13:35 2022] GET /identity/v3/role_assignments?user.id=db51971158a946c98a8ed716ecc5d72b&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=43aa4a9830054a09baa1f0aecfd125e2 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:35.866219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682367a-a38a-4e40-85cc-55f1fd970802 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:35.933628 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:35.979776 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1ea00-b8ce-4376-ad90-0f621c8eb250 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:35.980467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682367a-a38a-4e40-85cc-55f1fd970802 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:36.027899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fbbeb-ea36-4f31-a73f-be108b300fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:36.029059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fbbeb-ea36-4f31-a73f-be108b300fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:36.029059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fbbeb-ea36-4f31-a73f-be108b300fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:36.029249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7fbbeb-ea36-4f31-a73f-be108b300fc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:36.029322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e00f08-35b0-40e0-bdb6-a7bc93a3fb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:36.029908 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e00f08-35b0-40e0-bdb6-a7bc93a3fb2d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:36.030190 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e00f08-35b0-40e0-bdb6-a7bc93a3fb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:36.030464 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e00f08-35b0-40e0-bdb6-a7bc93a3fb2d None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:36.031120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e00f08-35b0-40e0-bdb6-a7bc93a3fb2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:36.031436 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e00f08-35b0-40e0-bdb6-a7bc93a3fb2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:36.033486 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e00f08-35b0-40e0-bdb6-a7bc93a3fb2d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:36.041412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7fbbeb-ea36-4f31-a73f-be108b300fc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:36.042921 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7fbbeb-ea36-4f31-a73f-be108b300fc7 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:36.050428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 287/576] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:36.051964 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e00f08-35b0-40e0-bdb6-a7bc93a3fb2d None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:36.058525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-8b7fbbeb-ea36-4f31-a73f-be108b300fc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:36.058525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:36.058525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:36.058525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:36.058525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:36.058525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:36.058525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:36.058525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:36.058525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:36.058525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:36.058525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:36.058525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:36.058525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:36.058525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:36.058525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:36.058525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:36.059832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:36.059832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:36.059832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:36.059832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:36.059832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:36.059832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:36.059832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:36.059832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:36.059832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:36.059832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:36.059832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:36.059832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:36.059832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:36.059832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:36.059832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:36.059832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:36.059832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:36.060770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:36.060770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:36.060770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:36.060770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 290/577] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:36.060770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fbbeb-ea36-4f31-a73f-be108b300fc7 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:36.077330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:36.167777 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e00f08-35b0-40e0-bdb6-a7bc93a3fb2d None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:36.170908 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7fbbeb-ea36-4f31-a73f-be108b300fc7 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:36.194435 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:36.207346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc4520-3170-468f-980d-f8b31e436ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:36.208366 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc4520-3170-468f-980d-f8b31e436ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:36.208851 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc4520-3170-468f-980d-f8b31e436ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:36.209183 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc4520-3170-468f-980d-f8b31e436ae0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:36.209735 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68feaff7-3eb9-464b-89dc-7a27e64e4864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:36.210323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68feaff7-3eb9-464b-89dc-7a27e64e4864 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:36.210603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68feaff7-3eb9-464b-89dc-7a27e64e4864 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:36.210875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68feaff7-3eb9-464b-89dc-7a27e64e4864 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:36.211531 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68feaff7-3eb9-464b-89dc-7a27e64e4864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:36.211833 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68feaff7-3eb9-464b-89dc-7a27e64e4864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:36.213261 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68feaff7-3eb9-464b-89dc-7a27e64e4864 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:36.222388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc4520-3170-468f-980d-f8b31e436ae0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:36.230536 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 291/578] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:36.230536 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68feaff7-3eb9-464b-89dc-7a27e64e4864 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:36.230766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-63bc4520-3170-468f-980d-f8b31e436ae0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:36.230766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:36.230766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:36.230766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:36.230766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:36.230766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:36.230766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:36.230766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:36.230766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:36.230766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:36.230766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:36.230766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:36.230766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:36.230766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:36.230766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:36.230766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:36.231633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:36.231633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:36.231633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:36.231633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:36.231633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:36.231633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:36.231633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:36.231633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:36.231633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:36.231633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:36.231633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:36.231633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:36.231633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:36.231633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:36.231633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:36.231633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:36.232505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:36.232505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:36.232505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:36.232505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:36.232505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:36.232505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:36.232505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:36.232505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:36.232505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:36.232505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:36.232505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:36.232505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:36.232505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 288/579] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:36.232505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc4520-3170-468f-980d-f8b31e436ae0 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:36.315372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:36.341332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68feaff7-3eb9-464b-89dc-7a27e64e4864 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:36.351870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc4520-3170-468f-980d-f8b31e436ae0 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:36.377887 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3db14e-3635-4823-bb5a-6a1c759e7120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:36.378636 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3db14e-3635-4823-bb5a-6a1c759e7120 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:36.378916 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3db14e-3635-4823-bb5a-6a1c759e7120 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:36.379187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3db14e-3635-4823-bb5a-6a1c759e7120 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:36.386440 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3db14e-3635-4823-bb5a-6a1c759e7120 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:36.387972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3db14e-3635-4823-bb5a-6a1c759e7120 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:36.391966 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2559fac-a900-4934-992f-2461df97a7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:36.392610 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2559fac-a900-4934-992f-2461df97a7ac None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:36.392888 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2559fac-a900-4934-992f-2461df97a7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:36.393157 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2559fac-a900-4934-992f-2461df97a7ac None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:36.393804 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2559fac-a900-4934-992f-2461df97a7ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:36.394106 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2559fac-a900-4934-992f-2461df97a7ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:36.395468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2559fac-a900-4934-992f-2461df97a7ac None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:36.399704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-ab3db14e-3635-4823-bb5a-6a1c759e7120 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:36.399704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:36.399704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:36.399704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:36.399704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:36.399704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:36.399704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:36.399704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:36.399704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:36.399704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:36.399704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:36.399704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:36.399704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:36.399704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:36.399704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:36.400615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:36.400615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:36.400615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:36.400615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:36.400615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:36.400615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:36.400615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:36.400615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:36.400615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:36.400615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:36.400615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:36.400615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:36.400615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:36.400615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:36.400615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:36.400615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:36.400615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:36.401603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:36.401603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:36.401603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:36.401603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:36.401603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:36.401603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:36.401603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:36.401603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:36.405820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 292/580] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:36.406643 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3db14e-3635-4823-bb5a-6a1c759e7120 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:36.408990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 289/581] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:36.409692 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2559fac-a900-4934-992f-2461df97a7ac None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:36.456649 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:36.515319 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3db14e-3635-4823-bb5a-6a1c759e7120 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:36.525929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2559fac-a900-4934-992f-2461df97a7ac None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:36.552172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1dd4d-2f14-4363-83b3-83a4f2c1da3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:36.552820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1dd4d-2f14-4363-83b3-83a4f2c1da3c None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:36.553096 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1dd4d-2f14-4363-83b3-83a4f2c1da3c None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:36.553363 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac1dd4d-2f14-4363-83b3-83a4f2c1da3c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:36.555605 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:36.564664 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1dd4d-2f14-4363-83b3-83a4f2c1da3c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:36.564942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a608a55-f9f6-4951-8510-768493371dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:36.565514 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a608a55-f9f6-4951-8510-768493371dda None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:36.565792 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a608a55-f9f6-4951-8510-768493371dda None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:36.566062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a608a55-f9f6-4951-8510-768493371dda None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:36.566154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac1dd4d-2f14-4363-83b3-83a4f2c1da3c None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:36.566728 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a608a55-f9f6-4951-8510-768493371dda None admin] RBAC: Adding query filter params (name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:36.567029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a608a55-f9f6-4951-8510-768493371dda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:36.568420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a608a55-f9f6-4951-8510-768493371dda None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:36.581408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 290/582] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:13:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:36.581627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-dac1dd4d-2f14-4363-83b3-83a4f2c1da3c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 19:13:36.581627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:36.581627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:36.581627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:36.581627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:36.581627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:36.581627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:36.581627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:36.581627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:36.581627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:36.581627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:36.581627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:36.581627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:36.581627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:36.581627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:36.581627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:36.582481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:36.583424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:36.583424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:36.583424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:36.583424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:36.583424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:36.583424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:36.583424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:36.583424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:36.583424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 19:13:36.583424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:36.583962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a608a55-f9f6-4951-8510-768493371dda None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:36.584073 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 293/583] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:36.584073 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1dd4d-2f14-4363-83b3-83a4f2c1da3c None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:36.700426 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a608a55-f9f6-4951-8510-768493371dda None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:36.703745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac1dd4d-2f14-4363-83b3-83a4f2c1da3c None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:36.707634 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:36.736394 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3458e-08f3-4959-b23d-4f55f840ebdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:36.737006 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3458e-08f3-4959-b23d-4f55f840ebdd None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:36.737283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3458e-08f3-4959-b23d-4f55f840ebdd None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:36.737624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3458e-08f3-4959-b23d-4f55f840ebdd None admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:36.738285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3458e-08f3-4959-b23d-4f55f840ebdd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:36.738589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3458e-08f3-4959-b23d-4f55f840ebdd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:36.739957 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab4196-03e0-48c1-b385-5544d4ba0876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:36.740106 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec3458e-08f3-4959-b23d-4f55f840ebdd None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:36.740645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab4196-03e0-48c1-b385-5544d4ba0876 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:36.740920 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab4196-03e0-48c1-b385-5544d4ba0876 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:36.741189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daab4196-03e0-48c1-b385-5544d4ba0876 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:36.749037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab4196-03e0-48c1-b385-5544d4ba0876 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:36.750521 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daab4196-03e0-48c1-b385-5544d4ba0876 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:36.757328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 291/584] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:36.757328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3458e-08f3-4959-b23d-4f55f840ebdd None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:36.767624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-daab4196-03e0-48c1-b385-5544d4ba0876 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:13:36.767624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:36.767624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:36.767624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:36.767624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:36.767624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:36.767624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:36.767624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:36.767624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:36.767624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:36.767624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:36.767624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:36.767624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:36.767624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:36.767624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:36.767624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:36.768720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:36.769672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:36.769672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:36.769672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:36.769672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:36.769672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:36.769672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:36.769672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:36.769672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:36.769672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:13:36.769672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:36.769672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 294/585] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Aug 7 19:13:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:36.769672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab4196-03e0-48c1-b385-5544d4ba0876 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:36.833225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:36.867151 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec3458e-08f3-4959-b23d-4f55f840ebdd None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:36.881202 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daab4196-03e0-48c1-b385-5544d4ba0876 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:36.906602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8a8f8-2ce9-4c4c-8297-92a0095a3dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:36.907264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8a8f8-2ce9-4c4c-8297-92a0095a3dbc None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:36.907548 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8a8f8-2ce9-4c4c-8297-92a0095a3dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:36.907823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8a8f8-2ce9-4c4c-8297-92a0095a3dbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:36.918256 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df2494-5e48-4373-a4ea-3768227d066a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:36.919024 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df2494-5e48-4373-a4ea-3768227d066a None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:36.919308 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df2494-5e48-4373-a4ea-3768227d066a None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:36.919579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df2494-5e48-4373-a4ea-3768227d066a None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:36.919680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b8a8f8-2ce9-4c4c-8297-92a0095a3dbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:36.920506 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df2494-5e48-4373-a4ea-3768227d066a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:36.920683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df2494-5e48-4373-a4ea-3768227d066a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:36.922213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df2494-5e48-4373-a4ea-3768227d066a None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:36.924902 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b8a8f8-2ce9-4c4c-8297-92a0095a3dbc None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:36.946487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 295/586] 10.222.0.17 () {56 vars in 1137 bytes} [Sun Aug 7 19:13:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:36.946487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df2494-5e48-4373-a4ea-3768227d066a None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:36.947440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-a5b8a8f8-2ce9-4c4c-8297-92a0095a3dbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:36.947440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:36.947440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:36.947440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:36.947440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:36.947440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:36.947440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:36.947440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:36.947440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:36.947440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:36.947440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:36.947440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:36.947440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:36.947440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:36.947440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:36.947440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:36.948658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:36.948658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:36.948658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:36.948658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:36.948658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:36.948658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:36.948658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:36.948658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:36.948658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:36.948658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:36.948658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:36.948658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:36.948658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:36.948658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:36.948658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:36.948658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:36.948658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:36.949532 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:36.949532 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:36.949532 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:36.952294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 292/587] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:36.953183 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8a8f8-2ce9-4c4c-8297-92a0095a3dbc None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:36.978737 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:37.063731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df2494-5e48-4373-a4ea-3768227d066a None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:37.075565 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8a8f8-2ce9-4c4c-8297-92a0095a3dbc None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:37.077072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:37.099378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f88db3-b885-4e84-b5e5-d13a008efe7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:37.100079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f88db3-b885-4e84-b5e5-d13a008efe7b None admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:37.100363 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f88db3-b885-4e84-b5e5-d13a008efe7b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:37.100634 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f88db3-b885-4e84-b5e5-d13a008efe7b None admin] PATH_INFO: `/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/a9daa1a3c7654dcfa5ccb03f467ce7d5/roles/43aa4a9830054a09baa1f0aecfd125e2` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:37.115196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f88db3-b885-4e84-b5e5-d13a008efe7b None admin] RBAC: Authorizing `identity:create_grant(project_id=37703d50cd7e4d9bbdc797a7a1466b66, user_id=a9daa1a3c7654dcfa5ccb03f467ce7d5, role_id=43aa4a9830054a09baa1f0aecfd125e2)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:37.116217 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15489b0-ee1b-4f19-aa04-27a0bd9db3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:37.116689 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f88db3-b885-4e84-b5e5-d13a008efe7b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:37.116870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15489b0-ee1b-4f19-aa04-27a0bd9db3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:37.117117 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15489b0-ee1b-4f19-aa04-27a0bd9db3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:37.117391 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15489b0-ee1b-4f19-aa04-27a0bd9db3f8 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:37.118129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15489b0-ee1b-4f19-aa04-27a0bd9db3f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:37.118437 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15489b0-ee1b-4f19-aa04-27a0bd9db3f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:37.120162 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15489b0-ee1b-4f19-aa04-27a0bd9db3f8 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:37.129802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 296/588] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:13:36 2022] PUT /identity/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/a9daa1a3c7654dcfa5ccb03f467ce7d5/roles/43aa4a9830054a09baa1f0aecfd125e2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:13:37.130855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f88db3-b885-4e84-b5e5-d13a008efe7b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:37.149817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 293/589] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Aug 7 19:13:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:37.149817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15489b0-ee1b-4f19-aa04-27a0bd9db3f8 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:37.239526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:37.268036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15489b0-ee1b-4f19-aa04-27a0bd9db3f8 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:37.294567 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f88db3-b885-4e84-b5e5-d13a008efe7b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:37.328700 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42210c3d-379b-4261-80ea-cfaa7ac562d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:37.329812 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42210c3d-379b-4261-80ea-cfaa7ac562d0 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:37.330328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42210c3d-379b-4261-80ea-cfaa7ac562d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:37.330758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42210c3d-379b-4261-80ea-cfaa7ac562d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:37.331420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42210c3d-379b-4261-80ea-cfaa7ac562d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:37.331725 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42210c3d-379b-4261-80ea-cfaa7ac562d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:37.334020 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42210c3d-379b-4261-80ea-cfaa7ac562d0 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:37.340959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba58af-07b5-4333-9dfe-234695ce6887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:37.341958 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba58af-07b5-4333-9dfe-234695ce6887 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:37.343074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba58af-07b5-4333-9dfe-234695ce6887 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:37.343606 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba58af-07b5-4333-9dfe-234695ce6887 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:37.347844 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 294/590] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:37.348776 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42210c3d-379b-4261-80ea-cfaa7ac562d0 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:37.357029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba58af-07b5-4333-9dfe-234695ce6887 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:37.358841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba58af-07b5-4333-9dfe-234695ce6887 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:37.380248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-cdba58af-07b5-4333-9dfe-234695ce6887 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:37.380248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:37.380248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:37.380248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:37.380248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:37.380248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:37.380248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:37.380248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:37.380248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:37.380248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:37.380248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:37.380248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:37.380248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:37.380248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:37.380248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:37.380248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:37.381287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:37.381287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:37.381287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:37.381287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:37.381287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:37.381287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:37.381287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:37.381287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:37.381287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:37.381287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:37.381287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:37.381287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:37.381287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:37.381287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:37.381287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:37.381287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:37.381287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:37.382169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:37.382169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:37.382169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:37.382169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 297/591] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:37.382837 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba58af-07b5-4333-9dfe-234695ce6887 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:37.487607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42210c3d-379b-4261-80ea-cfaa7ac562d0 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:37.488140 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba58af-07b5-4333-9dfe-234695ce6887 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:37.496410 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:37.525350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebb4b7-10a2-4879-acc8-4beeccf11052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:37.526001 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebb4b7-10a2-4879-acc8-4beeccf11052 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:37.526280 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebb4b7-10a2-4879-acc8-4beeccf11052 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:37.526553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aebb4b7-10a2-4879-acc8-4beeccf11052 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:37.533598 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec73bb-2cdd-4278-98c3-a2cec8df54f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:37.534556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec73bb-2cdd-4278-98c3-a2cec8df54f7 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:37.535032 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec73bb-2cdd-4278-98c3-a2cec8df54f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:37.535781 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec73bb-2cdd-4278-98c3-a2cec8df54f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:37.536569 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebb4b7-10a2-4879-acc8-4beeccf11052 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:37.538369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aebb4b7-10a2-4879-acc8-4beeccf11052 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:37.540646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:37.551151 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec73bb-2cdd-4278-98c3-a2cec8df54f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:37.552917 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec73bb-2cdd-4278-98c3-a2cec8df54f7 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:37.553102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-3aebb4b7-10a2-4879-acc8-4beeccf11052 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 19:13:37.553102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:37.553102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:37.553102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:37.553102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:37.553102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:37.553102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:37.553102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:37.553102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:37.553102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:37.553102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:37.553102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:37.553102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:37.553102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:37.553102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:37.553102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:37.553975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:37.554903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:37.554903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:37.554903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:37.554903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:37.554903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:37.554903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:37.554903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:37.554903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:37.554903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 19:13:37.554903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:37.559897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 298/592] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:37.561109 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebb4b7-10a2-4879-acc8-4beeccf11052 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:37.575941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-99ec73bb-2cdd-4278-98c3-a2cec8df54f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:37.575941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:37.575941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:37.575941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:37.575941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:37.575941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:37.575941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:37.575941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:37.575941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:37.575941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:37.575941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:37.575941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:37.575941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:37.575941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:37.575941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:37.575941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:37.576977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:37.576977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:37.576977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:37.576977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:37.576977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:37.576977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:37.576977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:37.576977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:37.576977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:37.576977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:37.576977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:37.576977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:37.576977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:37.576977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:37.576977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:37.576977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:37.576977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:37.577872 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:37.577872 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:37.577872 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:37.579946 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 295/593] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:37.581193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec73bb-2cdd-4278-98c3-a2cec8df54f7 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:37.694679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:37.723831 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec73bb-2cdd-4278-98c3-a2cec8df54f7 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:37.724633 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aebb4b7-10a2-4879-acc8-4beeccf11052 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:37.764231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e867c-063e-4c2a-b1be-f1669b136ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:37.765048 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e867c-063e-4c2a-b1be-f1669b136ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:37.765479 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e867c-063e-4c2a-b1be-f1669b136ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:37.765845 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e867c-063e-4c2a-b1be-f1669b136ae2 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:37.766691 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e867c-063e-4c2a-b1be-f1669b136ae2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:37.767134 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e867c-063e-4c2a-b1be-f1669b136ae2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:37.768568 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbbe43-6897-4ffb-85b4-c5f4c3e16f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:37.768816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e867c-063e-4c2a-b1be-f1669b136ae2 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:37.769527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbbe43-6897-4ffb-85b4-c5f4c3e16f0a None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:37.769949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbbe43-6897-4ffb-85b4-c5f4c3e16f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:37.770374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbbe43-6897-4ffb-85b4-c5f4c3e16f0a None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:37.771514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbbe43-6897-4ffb-85b4-c5f4c3e16f0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:37.771967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbbe43-6897-4ffb-85b4-c5f4c3e16f0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:37.774518 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbbe43-6897-4ffb-85b4-c5f4c3e16f0a None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:37.789363 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 296/594] 10.222.0.17 () {56 vars in 1139 bytes} [Sun Aug 7 19:13:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:37.790631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 299/595] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:37.792960 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e867c-063e-4c2a-b1be-f1669b136ae2 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:37.796068 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbbe43-6897-4ffb-85b4-c5f4c3e16f0a None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:37.855806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:37.914951 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbbe43-6897-4ffb-85b4-c5f4c3e16f0a None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:37.916142 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e867c-063e-4c2a-b1be-f1669b136ae2 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:37.953150 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe7275d-15e4-4bd0-831e-8a60a60b4269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:37.953795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7275d-15e4-4bd0-831e-8a60a60b4269 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:37.954068 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7275d-15e4-4bd0-831e-8a60a60b4269 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:37.954335 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe7275d-15e4-4bd0-831e-8a60a60b4269 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:37.956234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360f262-4540-42a7-a19f-98d74ee97ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:37.956842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360f262-4540-42a7-a19f-98d74ee97ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:37.957127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360f262-4540-42a7-a19f-98d74ee97ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:37.957401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360f262-4540-42a7-a19f-98d74ee97ac4 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:37.957500 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:37.958051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360f262-4540-42a7-a19f-98d74ee97ac4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:37.958508 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360f262-4540-42a7-a19f-98d74ee97ac4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:37.959783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360f262-4540-42a7-a19f-98d74ee97ac4 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:37.966882 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7275d-15e4-4bd0-831e-8a60a60b4269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:37.968384 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe7275d-15e4-4bd0-831e-8a60a60b4269 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:37.977918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 297/596] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:37.978794 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360f262-4540-42a7-a19f-98d74ee97ac4 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:37.985117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-dfe7275d-15e4-4bd0-831e-8a60a60b4269 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:37.985117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:37.985117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:37.985117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:37.985117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:37.985117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:37.985117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:37.985117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:37.985117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:37.985117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:37.985117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:37.985117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:37.985117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:37.985117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:37.985117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:37.985117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:37.986105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:37.986105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:37.986105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:37.986105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:37.986105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:37.986105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:37.986105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:37.986105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:37.986105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:37.986105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:37.986105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:37.986105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:37.986105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:37.986105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:37.986105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:37.986105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:37.986105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:37.986972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:37.986972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:37.986972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:37.990985 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 300/597] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:37.992127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb979445-c63e-4e41-9d69-87f7b6d43bb9 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:37.992413 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb979445-c63e-4e41-9d69-87f7b6d43bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:37.992687 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb979445-c63e-4e41-9d69-87f7b6d43bb9 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:37.993968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 301/598] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:37.994658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb979445-c63e-4e41-9d69-87f7b6d43bb9 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:38.099100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360f262-4540-42a7-a19f-98d74ee97ac4 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:38.106781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb979445-c63e-4e41-9d69-87f7b6d43bb9 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:38.107289 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:38.136299 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf6b8e-57ba-43ea-9789-e2e7b67ff846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:38.136996 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf6b8e-57ba-43ea-9789-e2e7b67ff846 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:38.137283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf6b8e-57ba-43ea-9789-e2e7b67ff846 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:38.137540 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf6b8e-57ba-43ea-9789-e2e7b67ff846 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:38.142748 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15ca50-2ee6-42c7-a06b-84f617bf424d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:38.143387 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15ca50-2ee6-42c7-a06b-84f617bf424d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:38.143753 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15ca50-2ee6-42c7-a06b-84f617bf424d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:38.144059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15ca50-2ee6-42c7-a06b-84f617bf424d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:38.144854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf6b8e-57ba-43ea-9789-e2e7b67ff846 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:38.146295 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf6b8e-57ba-43ea-9789-e2e7b67ff846 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:38.151015 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15ca50-2ee6-42c7-a06b-84f617bf424d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:38.152592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15ca50-2ee6-42c7-a06b-84f617bf424d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:38.158112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-eabf6b8e-57ba-43ea-9789-e2e7b67ff846 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:38.158112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:38.158112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:38.158112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:38.158112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:38.158112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:38.158112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:38.158112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:38.158112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:38.158112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:38.158112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:38.158112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:38.158112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:38.158112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:38.158112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:38.158976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:38.158976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:38.158976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:38.158976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:38.158976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:38.158976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:38.158976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:38.158976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:38.158976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:38.158976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:38.158976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:38.158976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:38.158976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:38.158976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:38.158976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:38.158976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:38.158976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:38.159900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:38.159900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:38.159900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:38.159900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:38.159900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:38.159900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:38.159900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:38.159900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:38.159900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 298/599] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:38.161082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf6b8e-57ba-43ea-9789-e2e7b67ff846 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:38.170038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-5e15ca50-2ee6-42c7-a06b-84f617bf424d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:38.170038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:38.170038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:38.170038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:38.170038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:38.170038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:38.170038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:38.170038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:38.170038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:38.170038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:38.170038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:38.170038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:38.170038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:38.170038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:38.170038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:38.170990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:38.170990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:38.170990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:38.170990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:38.170990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:38.170990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:38.170990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:38.170990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:38.170990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:38.170990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:38.170990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:38.170990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:38.170990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:38.170990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:38.170990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:38.170990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:38.170990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:38.177550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:38.177550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:38.177550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:38.177550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:38.177550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:38.177550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:38.177550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:38.177550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:38.177550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 302/600] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:38.177550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe3c26-d53c-46a8-8305-730e00ad966e None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:38.177550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe3c26-d53c-46a8-8305-730e00ad966e None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:38.178567 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbe3c26-d53c-46a8-8305-730e00ad966e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:38.193988 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-6cbe3c26-d53c-46a8-8305-730e00ad966e None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:38.195308 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-6cbe3c26-d53c-46a8-8305-730e00ad966e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:38.240335 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:38.272195 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 303/601] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:38.273139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe3c26-d53c-46a8-8305-730e00ad966e None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:38.287879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf6b8e-57ba-43ea-9789-e2e7b67ff846 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:38.327270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a7737-3d15-4c68-b281-45cf3e143e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:38.327901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a7737-3d15-4c68-b281-45cf3e143e7f None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:38.328221 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a7737-3d15-4c68-b281-45cf3e143e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:38.328523 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a7737-3d15-4c68-b281-45cf3e143e7f None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:38.329181 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a7737-3d15-4c68-b281-45cf3e143e7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:38.329485 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a7737-3d15-4c68-b281-45cf3e143e7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:38.330897 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a7737-3d15-4c68-b281-45cf3e143e7f None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:38.348823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 299/602] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:38.348823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a7737-3d15-4c68-b281-45cf3e143e7f None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:38.373635 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:38.395573 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbe3c26-d53c-46a8-8305-730e00ad966e None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:38.435419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f8212-b47b-46ec-b12e-fb674ae1f537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:38.436078 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f8212-b47b-46ec-b12e-fb674ae1f537 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:38.436358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f8212-b47b-46ec-b12e-fb674ae1f537 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:38.436635 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f8212-b47b-46ec-b12e-fb674ae1f537 None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:38.437340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f8212-b47b-46ec-b12e-fb674ae1f537 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:38.437643 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f8212-b47b-46ec-b12e-fb674ae1f537 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:38.439110 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f8212-b47b-46ec-b12e-fb674ae1f537 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:38.457508 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 304/603] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:38.457736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a7737-3d15-4c68-b281-45cf3e143e7f None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:38.457836 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f8212-b47b-46ec-b12e-fb674ae1f537 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:38.491933 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14aec0-4e11-4e49-8571-c3c7a9d59316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:38.492566 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:38.492678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14aec0-4e11-4e49-8571-c3c7a9d59316 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:38.492870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14aec0-4e11-4e49-8571-c3c7a9d59316 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:38.493148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14aec0-4e11-4e49-8571-c3c7a9d59316 None admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:38.493875 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14aec0-4e11-4e49-8571-c3c7a9d59316 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:38.494144 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14aec0-4e11-4e49-8571-c3c7a9d59316 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:38.495981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14aec0-4e11-4e49-8571-c3c7a9d59316 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:38.508829 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 300/604] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:38.509685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14aec0-4e11-4e49-8571-c3c7a9d59316 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:38.569622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f8212-b47b-46ec-b12e-fb674ae1f537 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:38.603116 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:38.608670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214c9c5-0116-4d6b-a3e7-25750892a68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:38.609373 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214c9c5-0116-4d6b-a3e7-25750892a68a None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:38.609655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214c9c5-0116-4d6b-a3e7-25750892a68a None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:38.609920 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214c9c5-0116-4d6b-a3e7-25750892a68a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:38.619979 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14aec0-4e11-4e49-8571-c3c7a9d59316 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:38.620858 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8214c9c5-0116-4d6b-a3e7-25750892a68a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:38.625611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-8214c9c5-0116-4d6b-a3e7-25750892a68a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:38.625611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:38.625611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:38.625611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:38.625611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:38.625611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:38.625611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:38.625611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:38.625611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:38.625611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:38.625611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:38.625611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:38.625611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:38.625611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:38.625611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:38.625611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:38.626562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:38.626562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:38.626562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:38.626562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:38.626562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:38.626562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:38.626562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:38.626562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:38.626562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:38.626562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:38.626562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:38.626562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:38.626562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:38.626562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:38.626562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:38.626562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:38.627420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:38.627420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:38.627420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:38.627420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:38.627420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:38.627420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:38.627420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:38.627420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:38.627420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:38.627420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:38.627420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:38.627420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:38.631741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 305/605] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:38.632489 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214c9c5-0116-4d6b-a3e7-25750892a68a None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:38.660808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac46f7b-47ba-4e15-8a1a-231829f1c12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:38.661611 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac46f7b-47ba-4e15-8a1a-231829f1c12a None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:38.661829 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac46f7b-47ba-4e15-8a1a-231829f1c12a None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:38.662113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac46f7b-47ba-4e15-8a1a-231829f1c12a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:38.670470 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac46f7b-47ba-4e15-8a1a-231829f1c12a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:38.671937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac46f7b-47ba-4e15-8a1a-231829f1c12a None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:38.684230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-dac46f7b-47ba-4e15-8a1a-231829f1c12a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:38.684230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:38.684230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:38.684230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:38.684230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:38.684230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:38.684230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:38.684230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:38.684230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:38.684230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:38.684230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:38.684230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:38.684230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:38.684230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:38.684230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:38.685322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:38.685322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:38.685322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:38.685322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:38.685322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:38.685322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:38.685322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:38.685322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:38.685322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:38.685322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:38.685322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:38.685322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:38.685322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:38.685322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:38.685322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:38.685322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:38.685322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:38.686520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:38.686520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:38.686520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:38.686520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:38.686520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:38.686520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:38.686520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:38.686520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:38.690163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 301/606] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:38.691005 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac46f7b-47ba-4e15-8a1a-231829f1c12a None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:38.740838 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:38.747468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214c9c5-0116-4d6b-a3e7-25750892a68a None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:38.784518 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3babf7-fa29-4fdb-9526-e737f5a10e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:38.785204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3babf7-fa29-4fdb-9526-e737f5a10e60 None admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:38.785530 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3babf7-fa29-4fdb-9526-e737f5a10e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:38.785858 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3babf7-fa29-4fdb-9526-e737f5a10e60 None admin] PATH_INFO: `/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/f83e5f0b1d5e4f57bbf283d420f080ec/roles/43aa4a9830054a09baa1f0aecfd125e2` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:38.799383 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac46f7b-47ba-4e15-8a1a-231829f1c12a None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:38.802674 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3babf7-fa29-4fdb-9526-e737f5a10e60 None admin] RBAC: Authorizing `identity:create_grant(project_id=37703d50cd7e4d9bbdc797a7a1466b66, user_id=f83e5f0b1d5e4f57bbf283d420f080ec, role_id=43aa4a9830054a09baa1f0aecfd125e2)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:38.804378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3babf7-fa29-4fdb-9526-e737f5a10e60 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:38.819894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 306/607] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:13:38 2022] PUT /identity/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/f83e5f0b1d5e4f57bbf283d420f080ec/roles/43aa4a9830054a09baa1f0aecfd125e2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:13:38.820990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3babf7-fa29-4fdb-9526-e737f5a10e60 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:38.849482 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94f5af-71c5-4be7-8714-03c1667b5f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:38.850499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94f5af-71c5-4be7-8714-03c1667b5f6c None admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:38.850843 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94f5af-71c5-4be7-8714-03c1667b5f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:38.851191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94f5af-71c5-4be7-8714-03c1667b5f6c None admin] PATH_INFO: `/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/a4946aec299f45bd8402cff3159d04d7/roles/43aa4a9830054a09baa1f0aecfd125e2` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:38.873006 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94f5af-71c5-4be7-8714-03c1667b5f6c None admin] RBAC: Authorizing `identity:create_grant(project_id=37703d50cd7e4d9bbdc797a7a1466b66, user_id=a4946aec299f45bd8402cff3159d04d7, role_id=43aa4a9830054a09baa1f0aecfd125e2)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:38.874848 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b94f5af-71c5-4be7-8714-03c1667b5f6c None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:38.879730 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:38.904323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 302/608] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:13:38 2022] PUT /identity/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/a4946aec299f45bd8402cff3159d04d7/roles/43aa4a9830054a09baa1f0aecfd125e2 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:13:38.905615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94f5af-71c5-4be7-8714-03c1667b5f6c None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:38.982197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3babf7-fa29-4fdb-9526-e737f5a10e60 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:39.071685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94aee32-232f-41cc-8760-abdadd5a72a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:39.072505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94aee32-232f-41cc-8760-abdadd5a72a7 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:39.072878 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94aee32-232f-41cc-8760-abdadd5a72a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:39.073561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94aee32-232f-41cc-8760-abdadd5a72a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:39.074085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94aee32-232f-41cc-8760-abdadd5a72a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:39.074494 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94aee32-232f-41cc-8760-abdadd5a72a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:39.076018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94aee32-232f-41cc-8760-abdadd5a72a7 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:39.097125 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b94f5af-71c5-4be7-8714-03c1667b5f6c None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:39.100026 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 307/609] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:13:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:39.111339 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94aee32-232f-41cc-8760-abdadd5a72a7 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:39.149598 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:39.160897 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc6cf6-99a3-44da-bfda-e7be56951b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:39.161653 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc6cf6-99a3-44da-bfda-e7be56951b71 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:39.162042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc6cf6-99a3-44da-bfda-e7be56951b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:39.162415 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc6cf6-99a3-44da-bfda-e7be56951b71 None admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:39.163486 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc6cf6-99a3-44da-bfda-e7be56951b71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:39.164081 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc6cf6-99a3-44da-bfda-e7be56951b71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:39.165683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc6cf6-99a3-44da-bfda-e7be56951b71 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:39.190640 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 303/610] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:39.196354 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc6cf6-99a3-44da-bfda-e7be56951b71 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:39.258131 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94aee32-232f-41cc-8760-abdadd5a72a7 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:39.270476 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:39.296539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf5c69-2b4f-4ae7-bb18-d6aa4dde660e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:39.297194 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf5c69-2b4f-4ae7-bb18-d6aa4dde660e None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:39.297488 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf5c69-2b4f-4ae7-bb18-d6aa4dde660e None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:39.297743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf5c69-2b4f-4ae7-bb18-d6aa4dde660e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:39.310352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf5c69-2b4f-4ae7-bb18-d6aa4dde660e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:39.312037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf5c69-2b4f-4ae7-bb18-d6aa4dde660e None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:39.320889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc6cf6-99a3-44da-bfda-e7be56951b71 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:39.327670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-8daf5c69-2b4f-4ae7-bb18-d6aa4dde660e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:39.327670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:39.327670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:39.327670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:39.327670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:39.327670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:39.327670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:39.327670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:39.327670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:39.327670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:39.327670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:39.327670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:39.327670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:39.327670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:39.327670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:39.327670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:39.328683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:39.328683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:39.328683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:39.328683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:39.328683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:39.328683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:39.328683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:39.328683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:39.328683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:39.328683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:39.328683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:39.328683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:39.328683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:39.328683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:39.328683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:39.328683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:39.328683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:39.329562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:39.329562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:39.329562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:39.333656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 308/611] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:39.334964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88cea3-47ae-4934-a54e-d50e34ef03d7 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:39.335555 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88cea3-47ae-4934-a54e-d50e34ef03d7 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:39.335665 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b88cea3-47ae-4934-a54e-d50e34ef03d7 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:39.337680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 309/612] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:39.338757 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88cea3-47ae-4934-a54e-d50e34ef03d7 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:39.361602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae38fc-c72c-49da-aa46-e1a24446f112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:39.362662 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae38fc-c72c-49da-aa46-e1a24446f112 None admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:39.363144 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae38fc-c72c-49da-aa46-e1a24446f112 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:39.363634 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae38fc-c72c-49da-aa46-e1a24446f112 None admin] PATH_INFO: `/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/08229e9400b14e8a9cc8e8c32dc48285/roles/43aa4a9830054a09baa1f0aecfd125e2` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:39.380447 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae38fc-c72c-49da-aa46-e1a24446f112 None admin] RBAC: Authorizing `identity:create_grant(project_id=37703d50cd7e4d9bbdc797a7a1466b66, user_id=08229e9400b14e8a9cc8e8c32dc48285, role_id=43aa4a9830054a09baa1f0aecfd125e2)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:39.382373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae38fc-c72c-49da-aa46-e1a24446f112 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:39.402130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 304/613] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:13:39 2022] PUT /identity/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/08229e9400b14e8a9cc8e8c32dc48285/roles/43aa4a9830054a09baa1f0aecfd125e2 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:13:39.404473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed912c28-5deb-4561-a89c-327c40b6e97a None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:39.405016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed912c28-5deb-4561-a89c-327c40b6e97a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:39.405581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed912c28-5deb-4561-a89c-327c40b6e97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:39.431408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:39.453426 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-ed912c28-5deb-4561-a89c-327c40b6e97a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:39.457202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ed912c28-5deb-4561-a89c-327c40b6e97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:39.515358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b88cea3-47ae-4934-a54e-d50e34ef03d7 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:39.575334 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e793f-fe59-4659-9e69-e2b9225c98c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:39.576655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e793f-fe59-4659-9e69-e2b9225c98c2 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:39.576934 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e793f-fe59-4659-9e69-e2b9225c98c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:39.577204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707e793f-fe59-4659-9e69-e2b9225c98c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:39.578373 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e793f-fe59-4659-9e69-e2b9225c98c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:39.578745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e793f-fe59-4659-9e69-e2b9225c98c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:39.580123 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707e793f-fe59-4659-9e69-e2b9225c98c2 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:39.596207 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 310/614] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:39.602077 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e793f-fe59-4659-9e69-e2b9225c98c2 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:39.602324 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 305/615] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:39.628024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed912c28-5deb-4561-a89c-327c40b6e97a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:39.703782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:39.744901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707e793f-fe59-4659-9e69-e2b9225c98c2 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:39.785471 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:39.813583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198d244-cdcb-4a17-a61a-1ac4321cdc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:39.814414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198d244-cdcb-4a17-a61a-1ac4321cdc87 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:39.815452 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198d244-cdcb-4a17-a61a-1ac4321cdc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:39.815980 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198d244-cdcb-4a17-a61a-1ac4321cdc87 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:39.829395 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198d244-cdcb-4a17-a61a-1ac4321cdc87 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:39.831665 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198d244-cdcb-4a17-a61a-1ac4321cdc87 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:39.834011 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed912c28-5deb-4561-a89c-327c40b6e97a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:39.855219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-7198d244-cdcb-4a17-a61a-1ac4321cdc87 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 19:13:39.855219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:39.855219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:39.855219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:39.855219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:39.855219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:39.855219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:39.855219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:39.855219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:39.855219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:39.855219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:39.855219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:39.855219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:39.855219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:39.855219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:39.855219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:39.856423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:39.857613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:39.857613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:39.857613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:39.857613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:39.857613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:39.857613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:39.857613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:39.857613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:39.857613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 19:13:39.857613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:39.863861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 311/616] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:13:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:39.867259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198d244-cdcb-4a17-a61a-1ac4321cdc87 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:39.905280 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9feb28-1b79-4aa6-94e3-6525deb94a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:39.906428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9feb28-1b79-4aa6-94e3-6525deb94a4e None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:39.907009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9feb28-1b79-4aa6-94e3-6525deb94a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:39.908314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9feb28-1b79-4aa6-94e3-6525deb94a4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:39.928085 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9feb28-1b79-4aa6-94e3-6525deb94a4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:39.937957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-6b9feb28-1b79-4aa6-94e3-6525deb94a4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:39.937957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:39.937957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:39.937957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:39.937957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:39.937957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:39.937957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:39.937957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:39.937957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:39.937957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:39.937957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:39.937957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:39.937957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:39.937957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:39.937957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:39.937957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:39.938941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:39.938941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:39.938941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:39.938941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:39.938941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:39.938941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:39.938941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:39.938941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:39.938941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:39.938941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:39.938941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:39.938941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:39.938941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:39.938941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:39.938941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:39.938941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:39.939793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:39.939793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:39.939793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:39.939793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:39.939793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:39.939793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:39.939793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:39.939793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:39.939793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:39.939793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:39.939793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:39.939793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:39.944998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 306/617] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:39.950050 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9feb28-1b79-4aa6-94e3-6525deb94a4e None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:40.059246 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198d244-cdcb-4a17-a61a-1ac4321cdc87 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:40.126412 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:40.128903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b406f-77ea-4d4f-892d-e74d9de825e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:40.129664 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b406f-77ea-4d4f-892d-e74d9de825e7 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:40.129950 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b406f-77ea-4d4f-892d-e74d9de825e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:40.130223 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b406f-77ea-4d4f-892d-e74d9de825e7 None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:40.130974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b406f-77ea-4d4f-892d-e74d9de825e7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:40.131283 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b406f-77ea-4d4f-892d-e74d9de825e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:40.132770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b406f-77ea-4d4f-892d-e74d9de825e7 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:40.134538 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9feb28-1b79-4aa6-94e3-6525deb94a4e None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:40.171487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44043b-95d9-4ebd-9dbe-ab3a9be27f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:40.176250 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 312/618] 10.222.0.17 () {56 vars in 1135 bytes} [Sun Aug 7 19:13:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:40.177197 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44043b-95d9-4ebd-9dbe-ab3a9be27f9f None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:40.177685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44043b-95d9-4ebd-9dbe-ab3a9be27f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:40.178161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44043b-95d9-4ebd-9dbe-ab3a9be27f9f None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:40.179202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44043b-95d9-4ebd-9dbe-ab3a9be27f9f None admin] RBAC: Adding query filter params (name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:40.179860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44043b-95d9-4ebd-9dbe-ab3a9be27f9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:40.181541 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44043b-95d9-4ebd-9dbe-ab3a9be27f9f None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:40.186234 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b406f-77ea-4d4f-892d-e74d9de825e7 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:40.201211 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 307/619] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:13:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:40.206898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44043b-95d9-4ebd-9dbe-ab3a9be27f9f None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:40.266413 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:40.378947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b406f-77ea-4d4f-892d-e74d9de825e7 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:40.414172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481f4f4-7283-4ab9-94d0-f1f3e0ae2880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:40.415186 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481f4f4-7283-4ab9-94d0-f1f3e0ae2880 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:40.415659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481f4f4-7283-4ab9-94d0-f1f3e0ae2880 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:40.416183 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481f4f4-7283-4ab9-94d0-f1f3e0ae2880 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:40.427147 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481f4f4-7283-4ab9-94d0-f1f3e0ae2880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:40.429035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481f4f4-7283-4ab9-94d0-f1f3e0ae2880 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:40.444872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-f481f4f4-7283-4ab9-94d0-f1f3e0ae2880 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:40.444872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:40.444872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:40.444872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:40.444872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:40.444872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:40.444872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:40.444872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:40.444872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:40.444872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:40.444872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:40.444872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:40.444872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:40.444872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:40.444872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:40.444872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:40.446029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:40.446029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:40.446029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:40.446029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:40.446029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:40.446029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:40.446029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:40.446029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:40.446029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:40.446029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:40.446029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:40.446029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:40.446029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:40.446029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:40.446029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:40.446029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:40.446029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:40.446919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:40.446919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:40.446919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:40.449146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 313/620] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:40.453994 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44043b-95d9-4ebd-9dbe-ab3a9be27f9f None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:40.454723 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481f4f4-7283-4ab9-94d0-f1f3e0ae2880 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:40.490683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbc948-8405-449a-bc8f-948c575a24f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:40.491335 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbc948-8405-449a-bc8f-948c575a24f8 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:40.491616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbc948-8405-449a-bc8f-948c575a24f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:40.491887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbc948-8405-449a-bc8f-948c575a24f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:40.504188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cbc948-8405-449a-bc8f-948c575a24f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:40.505804 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cbc948-8405-449a-bc8f-948c575a24f8 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:40.521716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-74cbc948-8405-449a-bc8f-948c575a24f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:40.521716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:40.521716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:40.521716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:40.521716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:40.521716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:40.521716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:40.521716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:40.521716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:40.521716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:40.521716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:40.521716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:40.521716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:40.521716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:40.521716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:40.521716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:40.522685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:40.522685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:40.522685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:40.522685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:40.522685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:40.522685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:40.522685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:40.522685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:40.522685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:40.522685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:40.522685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:40.522685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:40.522685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:40.522685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:40.522685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:40.522685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:40.522685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:40.523574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:40.523574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:40.523574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:40.523574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 308/621] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:40.532029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbc948-8405-449a-bc8f-948c575a24f8 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:40.576418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:40.652816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:40.668051 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481f4f4-7283-4ab9-94d0-f1f3e0ae2880 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:40.668867 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbc948-8405-449a-bc8f-948c575a24f8 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:40.714200 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf842a-0145-4e13-9f74-9c5f110e6d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:40.714920 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf842a-0145-4e13-9f74-9c5f110e6d8d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:40.715202 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf842a-0145-4e13-9f74-9c5f110e6d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:40.715476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf842a-0145-4e13-9f74-9c5f110e6d8d None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:40.716176 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf842a-0145-4e13-9f74-9c5f110e6d8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:40.716529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf842a-0145-4e13-9f74-9c5f110e6d8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:40.719434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf842a-0145-4e13-9f74-9c5f110e6d8d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:40.740245 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4160e-1c55-487c-8be0-907aeede35e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:40.740873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4160e-1c55-487c-8be0-907aeede35e5 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:40.741154 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4160e-1c55-487c-8be0-907aeede35e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:40.741428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4160e-1c55-487c-8be0-907aeede35e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:40.742088 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d4160e-1c55-487c-8be0-907aeede35e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:40.742395 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d4160e-1c55-487c-8be0-907aeede35e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:40.743771 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d4160e-1c55-487c-8be0-907aeede35e5 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:40.753241 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 314/622] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:40.754674 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf842a-0145-4e13-9f74-9c5f110e6d8d None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:40.768886 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 309/623] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:40.774593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4160e-1c55-487c-8be0-907aeede35e5 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:40.851492 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:40.893398 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4160e-1c55-487c-8be0-907aeede35e5 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:40.936745 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2cbf9-8d24-4762-b4db-a7b28556b373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:40.937414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2cbf9-8d24-4762-b4db-a7b28556b373 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:40.937694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2cbf9-8d24-4762-b4db-a7b28556b373 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:40.937965 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2cbf9-8d24-4762-b4db-a7b28556b373 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:40.951882 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2cbf9-8d24-4762-b4db-a7b28556b373 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:40.953404 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2cbf9-8d24-4762-b4db-a7b28556b373 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:40.979111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-4ac2cbf9-8d24-4762-b4db-a7b28556b373 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 07 19:13:40.979111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:40.979111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:40.979111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:40.979111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:40.979111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:40.979111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:40.979111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:40.979111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:40.979111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:40.979111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:40.979111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:40.979111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:40.979111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:40.979111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:40.979111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:40.980254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:40.981213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:40.981213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:40.981213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:40.981213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:40.981213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:40.981213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:40.981213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:40.981213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:40.981213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 07 19:13:40.981213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:40.982749 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 310/624] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:40.989411 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2cbf9-8d24-4762-b4db-a7b28556b373 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:41.042458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf842a-0145-4e13-9f74-9c5f110e6d8d None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:41.122838 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd428ae0-ef3c-4d48-adc5-3a93625f1b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:41.123503 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd428ae0-ef3c-4d48-adc5-3a93625f1b40 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:41.123781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd428ae0-ef3c-4d48-adc5-3a93625f1b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:41.124077 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd428ae0-ef3c-4d48-adc5-3a93625f1b40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:41.141712 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd428ae0-ef3c-4d48-adc5-3a93625f1b40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:41.143205 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd428ae0-ef3c-4d48-adc5-3a93625f1b40 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:41.165694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-cd428ae0-ef3c-4d48-adc5-3a93625f1b40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:41.165694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:41.165694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:41.165694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:41.165694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:41.165694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:41.165694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:41.165694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:41.165694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:41.165694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:41.165694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:41.165694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:41.165694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:41.165694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:41.165694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:41.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:41.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:41.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:41.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:41.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:41.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:41.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:41.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:41.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:41.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:41.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:41.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:41.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:41.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:41.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:41.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:41.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:41.167653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:41.167653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:41.167653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:41.167653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:41.167653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:41.167653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:41.167653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:41.167653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:41.167653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 315/625] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:41.174404 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd428ae0-ef3c-4d48-adc5-3a93625f1b40 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:41.186150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2cbf9-8d24-4762-b4db-a7b28556b373 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:41.205800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:41.218189 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:41.223945 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a196d6-ef51-44c0-9f89-1b8f07724e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:41.224660 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a196d6-ef51-44c0-9f89-1b8f07724e95 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:41.224927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a196d6-ef51-44c0-9f89-1b8f07724e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:41.225336 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a196d6-ef51-44c0-9f89-1b8f07724e95 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:41.226113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a196d6-ef51-44c0-9f89-1b8f07724e95 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:41.226505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a196d6-ef51-44c0-9f89-1b8f07724e95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:41.227925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a196d6-ef51-44c0-9f89-1b8f07724e95 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:41.254537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 311/626] 10.222.0.17 () {56 vars in 1137 bytes} [Sun Aug 7 19:13:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:41.260246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a196d6-ef51-44c0-9f89-1b8f07724e95 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:41.368020 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a196d6-ef51-44c0-9f89-1b8f07724e95 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:41.378302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd428ae0-ef3c-4d48-adc5-3a93625f1b40 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:41.403232 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc00edb-4045-4527-a3fc-01012749480b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:41.403837 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc00edb-4045-4527-a3fc-01012749480b None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:41.404147 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc00edb-4045-4527-a3fc-01012749480b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:41.404413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc00edb-4045-4527-a3fc-01012749480b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:41.422208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd281770-552b-4bfc-8564-d00a9628aef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:41.422208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd281770-552b-4bfc-8564-d00a9628aef1 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:41.422208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd281770-552b-4bfc-8564-d00a9628aef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:41.422208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd281770-552b-4bfc-8564-d00a9628aef1 None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:41.422884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd281770-552b-4bfc-8564-d00a9628aef1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:41.422884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd281770-552b-4bfc-8564-d00a9628aef1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:41.423208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd281770-552b-4bfc-8564-d00a9628aef1 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:41.442782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 316/627] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:41.442782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f9855-fd51-4492-9f4c-d3df9788fa25 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:41.442782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f9855-fd51-4492-9f4c-d3df9788fa25 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:41.442782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f9855-fd51-4492-9f4c-d3df9788fa25 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:41.442782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 317/628] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:41.442782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e75246-9303-489b-b66d-a7f23177d34d None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:41.443309 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e75246-9303-489b-b66d-a7f23177d34d None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:41.443309 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e75246-9303-489b-b66d-a7f23177d34d None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:41.444529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 318/629] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:41.446618 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e75246-9303-489b-b66d-a7f23177d34d None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:41.497228 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:41.518592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc00edb-4045-4527-a3fc-01012749480b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:41.520117 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc00edb-4045-4527-a3fc-01012749480b None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:41.545653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:41.581410 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e75246-9303-489b-b66d-a7f23177d34d None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:41.625344 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b2e53-3d57-420e-8141-fcd4fada4fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:41.625928 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b2e53-3d57-420e-8141-fcd4fada4fe9 None admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:41.626264 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b2e53-3d57-420e-8141-fcd4fada4fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:41.626587 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b2e53-3d57-420e-8141-fcd4fada4fe9 None admin] PATH_INFO: `/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/db51971158a946c98a8ed716ecc5d72b/roles/43aa4a9830054a09baa1f0aecfd125e2` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:41.643775 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b2e53-3d57-420e-8141-fcd4fada4fe9 None admin] RBAC: Authorizing `identity:create_grant(project_id=37703d50cd7e4d9bbdc797a7a1466b66, user_id=db51971158a946c98a8ed716ecc5d72b, role_id=43aa4a9830054a09baa1f0aecfd125e2)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:41.645772 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b2e53-3d57-420e-8141-fcd4fada4fe9 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:41.665548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 319/630] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:13:41 2022] PUT /identity/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/db51971158a946c98a8ed716ecc5d72b/roles/43aa4a9830054a09baa1f0aecfd125e2 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:13:41.669723 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a55548-54c4-4e99-8a44-c7aa4b467051 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:41.670179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a55548-54c4-4e99-8a44-c7aa4b467051 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:41.670519 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a55548-54c4-4e99-8a44-c7aa4b467051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:41.690624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 312/631] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 19:13:41.692468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3219b6e-4aec-416e-9506-24d0ff7394fe None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:41.693072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3219b6e-4aec-416e-9506-24d0ff7394fe None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:41.693601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3219b6e-4aec-416e-9506-24d0ff7394fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:41.700171 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-72a55548-54c4-4e99-8a44-c7aa4b467051 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:41.701408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-72a55548-54c4-4e99-8a44-c7aa4b467051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:41.717352 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-f3219b6e-4aec-416e-9506-24d0ff7394fe None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:41.720018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f3219b6e-4aec-416e-9506-24d0ff7394fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:41.794061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 320/632] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:41.795327 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a55548-54c4-4e99-8a44-c7aa4b467051 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:41.820433 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 313/633] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:41.822433 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe9b6c-4361-4129-95c0-f9e84b5df584 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:41.822835 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe9b6c-4361-4129-95c0-f9e84b5df584 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:41.823122 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe9b6c-4361-4129-95c0-f9e84b5df584 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:41.824904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 314/634] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:41.825535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe9b6c-4361-4129-95c0-f9e84b5df584 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:41.855575 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:41.864656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:41.919714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a55548-54c4-4e99-8a44-c7aa4b467051 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:41.945759 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe9b6c-4361-4129-95c0-f9e84b5df584 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:41.954933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aebcae-44fb-4294-850b-2b3c31199fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:41.955562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aebcae-44fb-4294-850b-2b3c31199fec None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:41.955834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aebcae-44fb-4294-850b-2b3c31199fec None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:41.956134 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aebcae-44fb-4294-850b-2b3c31199fec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:41.966233 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aebcae-44fb-4294-850b-2b3c31199fec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:41.967682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aebcae-44fb-4294-850b-2b3c31199fec None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:41.981782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e87f20-074b-4478-ab24-2c3202b6ee8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:41.982067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-67aebcae-44fb-4294-850b-2b3c31199fec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:41.982067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:41.982067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:41.982067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:41.982067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:41.982067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:41.982067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:41.982067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:41.982067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:41.982067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:41.982067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:41.982067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:41.982067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:41.982067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:41.982067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:41.982067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:41.983058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:41.983058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:41.983058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:41.983058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:41.983058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:41.983058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:41.983058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:41.983058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:41.983058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:41.983058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:41.983058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:41.983058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:41.983058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:41.983058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:41.983058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:41.983058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:41.983058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:41.989032 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:41.989032 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:41.989032 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:41.989257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e87f20-074b-4478-ab24-2c3202b6ee8c None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:41.989257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e87f20-074b-4478-ab24-2c3202b6ee8c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:41.989257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e87f20-074b-4478-ab24-2c3202b6ee8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:41.989461 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 321/635] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:41.989461 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21135986-0fa2-49b6-a0c2-541c79f7d953 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:41.989461 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21135986-0fa2-49b6-a0c2-541c79f7d953 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:41.989461 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21135986-0fa2-49b6-a0c2-541c79f7d953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:42.002047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e87f20-074b-4478-ab24-2c3202b6ee8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:42.003007 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-21135986-0fa2-49b6-a0c2-541c79f7d953 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:42.004217 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-21135986-0fa2-49b6-a0c2-541c79f7d953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:42.009795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-50e87f20-074b-4478-ab24-2c3202b6ee8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:42.009795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:42.009795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:42.009795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:42.009795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:42.009795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:42.009795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:42.009795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:42.009795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:42.009795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:42.009795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:42.009795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:42.009795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:42.009795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:42.009795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:42.009795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:42.010782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:42.010782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:42.010782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:42.010782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:42.010782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:42.010782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:42.010782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:42.010782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:42.010782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:42.010782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:42.010782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:42.010782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:42.010782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:42.010782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:42.010782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:42.010782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:42.016622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:42.016622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:42.016622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:42.016622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:42.016622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:42.016622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:42.016622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:42.016622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:42.016622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:42.016622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:42.016622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:42.016622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:42.016622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 315/636] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:42.016622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e87f20-074b-4478-ab24-2c3202b6ee8c None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:42.084644 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 322/637] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:42.085658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21135986-0fa2-49b6-a0c2-541c79f7d953 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:42.106139 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:42.129176 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:42.130972 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e87f20-074b-4478-ab24-2c3202b6ee8c None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:42.167910 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f1861-b898-427e-8dc6-861a4a4721b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:42.168589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f1861-b898-427e-8dc6-861a4a4721b4 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:42.168872 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f1861-b898-427e-8dc6-861a4a4721b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:42.169146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f1861-b898-427e-8dc6-861a4a4721b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:42.180246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f1861-b898-427e-8dc6-861a4a4721b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:42.185250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-5e5f1861-b898-427e-8dc6-861a4a4721b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:42.185250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:42.185250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:42.185250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:42.185250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:42.185250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:42.185250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:42.185250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:42.185250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:42.185250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:42.185250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:42.185250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:42.185250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:42.185250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:42.185250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:42.185250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:42.186208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:42.186208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:42.186208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:42.186208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:42.186208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:42.186208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:42.186208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:42.186208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:42.186208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:42.186208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:42.186208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:42.186208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:42.186208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:42.186208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:42.186208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:42.186208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:42.187491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:42.187491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:42.187491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:42.187491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:42.187491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:42.187491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:42.187491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:42.187491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:42.187491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:42.187491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:42.187491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:42.187491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:42.187491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 316/638] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:42.187491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f1861-b898-427e-8dc6-861a4a4721b4 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:42.210089 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21135986-0fa2-49b6-a0c2-541c79f7d953 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:42.244003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517e9b1-db28-4935-b783-b0051d1f6198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:42.244630 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517e9b1-db28-4935-b783-b0051d1f6198 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:42.244910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517e9b1-db28-4935-b783-b0051d1f6198 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:42.245176 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517e9b1-db28-4935-b783-b0051d1f6198 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:42.245874 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517e9b1-db28-4935-b783-b0051d1f6198 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:42.246162 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517e9b1-db28-4935-b783-b0051d1f6198 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:42.247540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6517e9b1-db28-4935-b783-b0051d1f6198 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:42.263477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 323/639] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:42.263477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517e9b1-db28-4935-b783-b0051d1f6198 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:42.295613 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f1861-b898-427e-8dc6-861a4a4721b4 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:42.332644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7faae-ad66-4445-8803-8ca7f1aee4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:42.333575 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7faae-ad66-4445-8803-8ca7f1aee4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:42.334005 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7faae-ad66-4445-8803-8ca7f1aee4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:42.334418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7faae-ad66-4445-8803-8ca7f1aee4b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:42.335404 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7faae-ad66-4445-8803-8ca7f1aee4b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:42.335865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7faae-ad66-4445-8803-8ca7f1aee4b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:42.337504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7faae-ad66-4445-8803-8ca7f1aee4b0 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:42.344627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:42.352533 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 317/640] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:13:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:42.353100 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:42.353556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7faae-ad66-4445-8803-8ca7f1aee4b0 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:42.364463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517e9b1-db28-4935-b783-b0051d1f6198 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:42.399456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f197442-fe97-4130-b6d2-654fc1e6b611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:42.400116 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f197442-fe97-4130-b6d2-654fc1e6b611 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:42.400396 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f197442-fe97-4130-b6d2-654fc1e6b611 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:42.400670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f197442-fe97-4130-b6d2-654fc1e6b611 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:42.411439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f197442-fe97-4130-b6d2-654fc1e6b611 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:42.416562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-8f197442-fe97-4130-b6d2-654fc1e6b611 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:42.416562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:42.416562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:42.416562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:42.416562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:42.416562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:42.416562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:42.416562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:42.416562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:42.416562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:42.416562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:42.416562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:42.416562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:42.416562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:42.416562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:42.416562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:42.417559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:42.417559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:42.417559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:42.417559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:42.417559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:42.417559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:42.417559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:42.417559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:42.417559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:42.417559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:42.417559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:42.417559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:42.417559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:42.417559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:42.417559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:42.417559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:42.418415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:42.418415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:42.418415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:42.418415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:42.418415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:42.418415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:42.418415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:42.418415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:42.418415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:42.418415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:42.418415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:42.418415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:42.418415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 324/641] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:42.419145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f197442-fe97-4130-b6d2-654fc1e6b611 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:42.466988 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7faae-ad66-4445-8803-8ca7f1aee4b0 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:42.503582 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c07713-c313-44f6-bede-6cfd200db509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:42.504243 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07713-c313-44f6-bede-6cfd200db509 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:42.504538 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07713-c313-44f6-bede-6cfd200db509 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:42.504813 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c07713-c313-44f6-bede-6cfd200db509 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:42.505475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c07713-c313-44f6-bede-6cfd200db509 None admin] RBAC: Adding query filter params (name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:42.505785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c07713-c313-44f6-bede-6cfd200db509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:42.507157 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c07713-c313-44f6-bede-6cfd200db509 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:42.515917 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 318/642] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:13:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:42.521053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c07713-c313-44f6-bede-6cfd200db509 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:42.524431 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f197442-fe97-4130-b6d2-654fc1e6b611 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:42.561809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43079fac-cd8b-4072-989a-c2ceaeb45c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:42.562437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43079fac-cd8b-4072-989a-c2ceaeb45c5f None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:42.562732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43079fac-cd8b-4072-989a-c2ceaeb45c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:42.562987 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43079fac-cd8b-4072-989a-c2ceaeb45c5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:42.570797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43079fac-cd8b-4072-989a-c2ceaeb45c5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:42.572373 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43079fac-cd8b-4072-989a-c2ceaeb45c5f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:42.583781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-43079fac-cd8b-4072-989a-c2ceaeb45c5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:42.583781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:42.583781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:42.583781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:42.583781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:42.583781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:42.583781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:42.583781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:42.583781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:42.583781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:42.583781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:42.583781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:42.583781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:42.583781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:42.583781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:42.591695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:42.591695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:42.591695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:42.591695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:42.591695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:42.591695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:42.591695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:42.591695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:42.591695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:42.591695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:42.591695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:42.591695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:42.591695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:42.591695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:42.591695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:42.591695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:42.591695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:42.592830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:42.592830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:42.592830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:42.592830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:42.592830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:42.592830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:42.592830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:42.592830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:42.592830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 325/643] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:42.592830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43079fac-cd8b-4072-989a-c2ceaeb45c5f None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:42.597223 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:42.611327 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:42.646811 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c07713-c313-44f6-bede-6cfd200db509 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:42.687042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4ff1e-f7fa-474a-b671-b90ff16b50c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:42.687848 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4ff1e-f7fa-474a-b671-b90ff16b50c6 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:42.688232 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4ff1e-f7fa-474a-b671-b90ff16b50c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:42.688577 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4ff1e-f7fa-474a-b671-b90ff16b50c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:42.700704 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4ff1e-f7fa-474a-b671-b90ff16b50c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:42.702246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4ff1e-f7fa-474a-b671-b90ff16b50c6 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:42.710114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43079fac-cd8b-4072-989a-c2ceaeb45c5f None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:42.718112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-50d4ff1e-f7fa-474a-b671-b90ff16b50c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:42.718112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:42.718112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:42.718112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:42.718112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:42.718112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:42.718112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:42.718112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:42.718112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:42.718112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:42.718112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:42.718112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:42.718112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:42.718112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:42.718112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:42.718112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:42.719263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:42.719263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:42.719263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:42.719263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:42.719263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:42.719263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:42.719263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:42.719263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:42.719263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:42.719263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:42.719263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:42.719263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:42.719263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:42.719263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:42.719263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:42.719263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:42.719263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:42.720786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:42.720786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:42.720786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:42.721164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 319/644] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:42.722138 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4ff1e-f7fa-474a-b671-b90ff16b50c6 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:42.752496 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f662ca-73a0-436d-adb5-d935d2780304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:42.753160 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f662ca-73a0-436d-adb5-d935d2780304 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:42.753433 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f662ca-73a0-436d-adb5-d935d2780304 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:42.754187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f662ca-73a0-436d-adb5-d935d2780304 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:42.755002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f662ca-73a0-436d-adb5-d935d2780304 None admin] RBAC: Adding query filter params (name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:42.755210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f662ca-73a0-436d-adb5-d935d2780304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:42.756706 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f662ca-73a0-436d-adb5-d935d2780304 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:42.765565 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 326/645] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:13:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:42.766492 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f662ca-73a0-436d-adb5-d935d2780304 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:42.849976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4ff1e-f7fa-474a-b671-b90ff16b50c6 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:42.852816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:42.865041 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:42.878950 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f662ca-73a0-436d-adb5-d935d2780304 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:42.887430 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2802d-34b1-4c74-913c-da0f83791553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:42.888104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2802d-34b1-4c74-913c-da0f83791553 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:42.888388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2802d-34b1-4c74-913c-da0f83791553 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:42.888666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2802d-34b1-4c74-913c-da0f83791553 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:42.899385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2802d-34b1-4c74-913c-da0f83791553 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:42.900920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2802d-34b1-4c74-913c-da0f83791553 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:42.912971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ac7cf-ae31-4233-be48-4123aa13bac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:42.913695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ac7cf-ae31-4233-be48-4123aa13bac4 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:42.914065 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ac7cf-ae31-4233-be48-4123aa13bac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:42.914428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ac7cf-ae31-4233-be48-4123aa13bac4 None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:42.915112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-75c2802d-34b1-4c74-913c-da0f83791553 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:42.915112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:42.915112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:42.915112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:42.915112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:42.915112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:42.915112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:42.915112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:42.915112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:42.915112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:42.915112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:42.915112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:42.915112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:42.915112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:42.915112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:42.915112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:42.916242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:42.916242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:42.916242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:42.916242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:42.916242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:42.916242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:42.916242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:42.916242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:42.916242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:42.916242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:42.916242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:42.916242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:42.916242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:42.916242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:42.916242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:42.916242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:42.916242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:42.917339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:42.917339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:42.917339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:42.917580 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ac7cf-ae31-4233-be48-4123aa13bac4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:42.917700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ac7cf-ae31-4233-be48-4123aa13bac4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:42.919252 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ac7cf-ae31-4233-be48-4123aa13bac4 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:42.920940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 320/646] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:42.921744 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2802d-34b1-4c74-913c-da0f83791553 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:42.939287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 327/647] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:42.940416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ac7cf-ae31-4233-be48-4123aa13bac4 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:43.052350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ac7cf-ae31-4233-be48-4123aa13bac4 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:43.057994 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2802d-34b1-4c74-913c-da0f83791553 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:43.086823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d67ca0-e18f-4d78-977d-3999ee5456b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:43.087596 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d67ca0-e18f-4d78-977d-3999ee5456b0 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:43.087966 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d67ca0-e18f-4d78-977d-3999ee5456b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:43.088265 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d67ca0-e18f-4d78-977d-3999ee5456b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:43.090355 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:43.092636 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a006e-92bd-4237-a9d1-b5a50929e813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:43.093455 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a006e-92bd-4237-a9d1-b5a50929e813 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:43.093733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a006e-92bd-4237-a9d1-b5a50929e813 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:43.094002 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a006e-92bd-4237-a9d1-b5a50929e813 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:43.094648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a006e-92bd-4237-a9d1-b5a50929e813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:43.094947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a006e-92bd-4237-a9d1-b5a50929e813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:43.096332 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a006e-92bd-4237-a9d1-b5a50929e813 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:43.101076 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d67ca0-e18f-4d78-977d-3999ee5456b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:43.102521 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d67ca0-e18f-4d78-977d-3999ee5456b0 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:43.113044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 321/648] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:43.113044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a006e-92bd-4237-a9d1-b5a50929e813 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:43.122421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-31d67ca0-e18f-4d78-977d-3999ee5456b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:43.122421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:43.122421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:43.122421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:43.122421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:43.122421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:43.122421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:43.122421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:43.122421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:43.122421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:43.122421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:43.122421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:43.122421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:43.122421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:43.122421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:43.122421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:43.123405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:43.123405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:43.123405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:43.123405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:43.123405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:43.123405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:43.123405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:43.123405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:43.123405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:43.123405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:43.123405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:43.123405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:43.123405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:43.123405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:43.123405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:43.123405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:43.123405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:43.124675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:43.124675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:43.124675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:43.124675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 328/649] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:43.124675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d67ca0-e18f-4d78-977d-3999ee5456b0 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:43.145066 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:43.221445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a006e-92bd-4237-a9d1-b5a50929e813 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:43.232889 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d67ca0-e18f-4d78-977d-3999ee5456b0 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:43.259531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1f70a-c6e6-4fc1-8201-5d61459e2c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:43.260436 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1f70a-c6e6-4fc1-8201-5d61459e2c90 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:43.260927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1f70a-c6e6-4fc1-8201-5d61459e2c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:43.261542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1f70a-c6e6-4fc1-8201-5d61459e2c90 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:43.262464 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1f70a-c6e6-4fc1-8201-5d61459e2c90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:43.262941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1f70a-c6e6-4fc1-8201-5d61459e2c90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:43.264674 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1f70a-c6e6-4fc1-8201-5d61459e2c90 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:43.266758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb728379-5056-4253-9a55-208534bbfc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:43.267371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb728379-5056-4253-9a55-208534bbfc2f None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:43.267645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb728379-5056-4253-9a55-208534bbfc2f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:43.267933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb728379-5056-4253-9a55-208534bbfc2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:43.268682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb728379-5056-4253-9a55-208534bbfc2f None admin] RBAC: Adding query filter params (role.id=43aa4a9830054a09baa1f0aecfd125e2, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=a9daa1a3c7654dcfa5ccb03f467ce7d5) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:43.268987 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb728379-5056-4253-9a55-208534bbfc2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:43.270405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb728379-5056-4253-9a55-208534bbfc2f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:43.276395 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 329/650] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:13:43 2022] GET /identity/v3/role_assignments?user.id=a9daa1a3c7654dcfa5ccb03f467ce7d5&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=43aa4a9830054a09baa1f0aecfd125e2 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:43.277256 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb728379-5056-4253-9a55-208534bbfc2f None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:43.278138 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 322/651] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:43.279101 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1f70a-c6e6-4fc1-8201-5d61459e2c90 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:43.348934 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:43.395840 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:43.416868 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1f70a-c6e6-4fc1-8201-5d61459e2c90 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:43.417348 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb728379-5056-4253-9a55-208534bbfc2f None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:43.455365 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a9f21-6b0d-4cff-8066-9ebbfdfe85da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:43.456344 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a9f21-6b0d-4cff-8066-9ebbfdfe85da None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:43.456777 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a9f21-6b0d-4cff-8066-9ebbfdfe85da None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:43.457175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03a9f21-6b0d-4cff-8066-9ebbfdfe85da None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:43.473291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c0b75-d206-4019-aa12-10c5e8e1f23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:43.473557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c0b75-d206-4019-aa12-10c5e8e1f23b None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:43.473933 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c0b75-d206-4019-aa12-10c5e8e1f23b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:43.474308 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c0b75-d206-4019-aa12-10c5e8e1f23b None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:43.475093 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c0b75-d206-4019-aa12-10c5e8e1f23b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:43.475504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c0b75-d206-4019-aa12-10c5e8e1f23b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:43.477365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c0b75-d206-4019-aa12-10c5e8e1f23b None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:43.478467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a9f21-6b0d-4cff-8066-9ebbfdfe85da None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:43.480557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03a9f21-6b0d-4cff-8066-9ebbfdfe85da None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:43.496763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-a03a9f21-6b0d-4cff-8066-9ebbfdfe85da None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 19:13:43.496763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:43.496763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:43.496763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:43.496763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:43.496763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:43.496763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:43.496763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:43.496763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:43.496763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:43.496763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:43.496763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:43.496763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:43.496763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:43.496763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:43.496763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:43.497796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:43.498794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:43.498794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:43.498794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:43.498794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:43.498794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:43.498794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:43.498794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:43.498794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:43.498794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 19:13:43.498794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:43.499418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 323/652] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:43.500512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c0b75-d206-4019-aa12-10c5e8e1f23b None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:43.501320 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 330/653] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:43.502293 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a9f21-6b0d-4cff-8066-9ebbfdfe85da None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:43.644860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c0b75-d206-4019-aa12-10c5e8e1f23b None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:43.645051 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03a9f21-6b0d-4cff-8066-9ebbfdfe85da None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:43.664602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:43.687246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbda207-9005-475f-9beb-c08c90e28fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:43.687887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbda207-9005-475f-9beb-c08c90e28fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:43.688191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbda207-9005-475f-9beb-c08c90e28fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:43.688460 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbda207-9005-475f-9beb-c08c90e28fd5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:43.688681 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef74271-933e-41bd-a77c-09032fac48d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:43.689629 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef74271-933e-41bd-a77c-09032fac48d1 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:43.689997 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef74271-933e-41bd-a77c-09032fac48d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:43.690367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef74271-933e-41bd-a77c-09032fac48d1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:43.696460 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbda207-9005-475f-9beb-c08c90e28fd5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:43.697920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbda207-9005-475f-9beb-c08c90e28fd5 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:43.699233 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef74271-933e-41bd-a77c-09032fac48d1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:43.701010 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef74271-933e-41bd-a77c-09032fac48d1 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:43.711682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-abbda207-9005-475f-9beb-c08c90e28fd5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:13:43.711682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:43.711682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:43.711682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:43.711682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:43.711682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:43.711682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:43.711682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:43.711682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:43.711682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:43.711682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:43.711682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:43.711682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:43.711682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:43.711682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:43.711682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:43.712977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:43.714140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:43.714140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:43.714140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:43.714140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:43.714140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:43.714140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:43.714140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:43.714140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:43.714140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:13:43.714140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:43.714924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-2ef74271-933e-41bd-a77c-09032fac48d1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 19:13:43.714924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:43.714924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:43.714924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:43.714924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:43.714924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:43.714924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:43.714924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:43.714924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:43.714924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:43.714924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:43.714924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:43.714924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:43.714924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:43.714924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:43.714924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:43.715800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:43.717014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:43.717014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:43.717014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:43.717014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:43.717014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:43.717014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:43.717014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:43.717014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:43.717014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 19:13:43.717014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:43.719884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 324/655] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Aug 7 19:13:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:43.720385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 331/655] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:43.720844 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbda207-9005-475f-9beb-c08c90e28fd5 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:43.721579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272c7ee-2a15-439e-8bd4-cb866212aba2 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:43.721866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272c7ee-2a15-439e-8bd4-cb866212aba2 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:43.722143 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272c7ee-2a15-439e-8bd4-cb866212aba2 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:43.724041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 332/656] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:43.724388 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272c7ee-2a15-439e-8bd4-cb866212aba2 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:43.731568 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:43.841453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbda207-9005-475f-9beb-c08c90e28fd5 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:43.848866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272c7ee-2a15-439e-8bd4-cb866212aba2 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:43.883101 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0814fe-1fc2-4233-a364-f620833a680a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:43.884177 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0814fe-1fc2-4233-a364-f620833a680a None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:43.884565 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0814fe-1fc2-4233-a364-f620833a680a None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:43.885014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0814fe-1fc2-4233-a364-f620833a680a None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:43.886116 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0814fe-1fc2-4233-a364-f620833a680a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:43.886704 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0814fe-1fc2-4233-a364-f620833a680a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:43.889002 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0814fe-1fc2-4233-a364-f620833a680a None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:43.890989 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3c7eb-c065-4656-b3d7-3d84352a4922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:43.892127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3c7eb-c065-4656-b3d7-3d84352a4922 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:43.892265 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3c7eb-c065-4656-b3d7-3d84352a4922 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:43.892537 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3c7eb-c065-4656-b3d7-3d84352a4922 None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:43.894178 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3c7eb-c065-4656-b3d7-3d84352a4922 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:43.894562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3c7eb-c065-4656-b3d7-3d84352a4922 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:43.896154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3c7eb-c065-4656-b3d7-3d84352a4922 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:43.910838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 325/657] 10.222.0.17 () {56 vars in 1137 bytes} [Sun Aug 7 19:13:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:43.911496 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0814fe-1fc2-4233-a364-f620833a680a None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:43.924254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 333/658] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Aug 7 19:13:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:43.924254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da30075-cc86-4062-8a37-3f8fb5080988 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:43.924254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da30075-cc86-4062-8a37-3f8fb5080988 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:43.924254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da30075-cc86-4062-8a37-3f8fb5080988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:43.942897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-7da30075-cc86-4062-8a37-3f8fb5080988 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:43.944264 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-7da30075-cc86-4062-8a37-3f8fb5080988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:43.971580 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:43.992037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:44.036076 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 334/659] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:44.036076 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da30075-cc86-4062-8a37-3f8fb5080988 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:44.044924 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0814fe-1fc2-4233-a364-f620833a680a None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:44.084523 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206ef9c-3662-46e5-860e-c5f36e2cd529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:44.085231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206ef9c-3662-46e5-860e-c5f36e2cd529 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:44.085564 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206ef9c-3662-46e5-860e-c5f36e2cd529 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:44.085909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206ef9c-3662-46e5-860e-c5f36e2cd529 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:44.086655 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206ef9c-3662-46e5-860e-c5f36e2cd529 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:44.086957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206ef9c-3662-46e5-860e-c5f36e2cd529 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:44.088445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206ef9c-3662-46e5-860e-c5f36e2cd529 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:44.107071 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 326/660] 10.222.0.17 () {56 vars in 1139 bytes} [Sun Aug 7 19:13:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:44.107931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206ef9c-3662-46e5-860e-c5f36e2cd529 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:44.152540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da30075-cc86-4062-8a37-3f8fb5080988 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:44.191679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab9d99-c34f-429f-a05c-856d4ee0c816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:44.192351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab9d99-c34f-429f-a05c-856d4ee0c816 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:44.192633 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab9d99-c34f-429f-a05c-856d4ee0c816 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:44.192907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab9d99-c34f-429f-a05c-856d4ee0c816 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:44.222702 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:44.223786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206ef9c-3662-46e5-860e-c5f36e2cd529 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:44.255872 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:44.275584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d2a27-16eb-4553-9b61-58d4d43c76c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:44.275584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d2a27-16eb-4553-9b61-58d4d43c76c7 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:44.275584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d2a27-16eb-4553-9b61-58d4d43c76c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:44.275584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d2a27-16eb-4553-9b61-58d4d43c76c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:44.279147 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab9d99-c34f-429f-a05c-856d4ee0c816 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:44.280667 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab9d99-c34f-429f-a05c-856d4ee0c816 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:44.344903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d2a27-16eb-4553-9b61-58d4d43c76c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:44.346389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d2a27-16eb-4553-9b61-58d4d43c76c7 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:44.386362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 335/661] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:44.387221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab9d99-c34f-429f-a05c-856d4ee0c816 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:44.449806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:44.457464 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 327/662] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:44.463032 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d2a27-16eb-4553-9b61-58d4d43c76c7 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:44.487973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:44.495792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab9d99-c34f-429f-a05c-856d4ee0c816 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:44.534169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29132b84-b529-4543-b2c9-78bc78542edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:44.534811 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29132b84-b529-4543-b2c9-78bc78542edd None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:44.535087 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29132b84-b529-4543-b2c9-78bc78542edd None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:44.535362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29132b84-b529-4543-b2c9-78bc78542edd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:44.546579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29132b84-b529-4543-b2c9-78bc78542edd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:44.551354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-29132b84-b529-4543-b2c9-78bc78542edd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:44.551354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:44.551354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:44.551354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:44.551354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:44.551354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:44.551354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:44.551354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:44.551354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:44.551354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:44.551354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:44.551354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:44.551354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:44.551354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:44.551354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:44.551354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:44.552326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:44.552326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:44.552326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:44.552326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:44.552326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:44.552326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:44.552326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:44.552326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:44.552326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:44.552326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:44.552326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:44.552326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:44.552326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:44.552326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:44.552326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:44.552326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:44.553173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:44.553173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:44.553173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:44.553173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:44.553173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:44.553173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:44.553173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:44.553173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:44.553173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:44.553173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:44.553173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:44.553173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:44.553173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 336/663] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:44.554232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29132b84-b529-4543-b2c9-78bc78542edd None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:44.577757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d2a27-16eb-4553-9b61-58d4d43c76c7 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:44.619839 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8545d0e-1aa9-4994-9df8-2cf6e09de547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:44.620512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8545d0e-1aa9-4994-9df8-2cf6e09de547 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:44.620990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8545d0e-1aa9-4994-9df8-2cf6e09de547 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:44.621164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8545d0e-1aa9-4994-9df8-2cf6e09de547 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:44.672385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29132b84-b529-4543-b2c9-78bc78542edd None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:44.692194 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:44.703121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8545d0e-1aa9-4994-9df8-2cf6e09de547 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:44.704569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8545d0e-1aa9-4994-9df8-2cf6e09de547 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:44.710520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ac4fc-16a7-4836-8edf-8210031491aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:44.711170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ac4fc-16a7-4836-8edf-8210031491aa None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:44.711445 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ac4fc-16a7-4836-8edf-8210031491aa None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:44.711713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ac4fc-16a7-4836-8edf-8210031491aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:44.723829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ac4fc-16a7-4836-8edf-8210031491aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:44.725396 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ac4fc-16a7-4836-8edf-8210031491aa None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:44.729507 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:44.741514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-5b4ac4fc-16a7-4836-8edf-8210031491aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:44.741514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:44.741514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:44.741514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:44.741514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:44.741514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:44.741514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:44.741514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:44.741514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:44.741514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:44.741514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:44.741514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:44.741514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:44.741514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:44.741514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:44.741514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:44.742526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:44.742526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:44.742526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:44.742526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:44.742526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:44.742526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:44.742526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:44.742526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:44.742526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:44.742526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:44.742526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:44.742526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:44.742526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:44.742526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:44.742526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:44.742526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:44.742526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:44.743401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:44.743401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:44.743401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:44.743401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 337/664] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:44.743621 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ac4fc-16a7-4836-8edf-8210031491aa None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:44.817906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 328/665] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:44.817906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8545d0e-1aa9-4994-9df8-2cf6e09de547 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:44.852480 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ac4fc-16a7-4836-8edf-8210031491aa None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:44.887452 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e9432-d820-48ef-a8c0-2419ecce3464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:44.888114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e9432-d820-48ef-a8c0-2419ecce3464 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:44.888397 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e9432-d820-48ef-a8c0-2419ecce3464 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:44.888670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e9432-d820-48ef-a8c0-2419ecce3464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:44.900545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e9432-d820-48ef-a8c0-2419ecce3464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:44.902101 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e9432-d820-48ef-a8c0-2419ecce3464 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:44.917255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-d44e9432-d820-48ef-a8c0-2419ecce3464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:44.917255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:44.917255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:44.917255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:44.917255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:44.917255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:44.917255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:44.917255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:44.917255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:44.917255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:44.917255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:44.917255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:44.917255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:44.917255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:44.917255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:44.917255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:44.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:44.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:44.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:44.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:44.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:44.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:44.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:44.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:44.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:44.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:44.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:44.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:44.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:44.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:44.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:44.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:44.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:44.919250 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:44.919250 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:44.919250 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:44.919250 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 338/666] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:44.919492 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e9432-d820-48ef-a8c0-2419ecce3464 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:44.928931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8545d0e-1aa9-4994-9df8-2cf6e09de547 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:44.947322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:44.955810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:44.973174 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727a90c-4596-40d5-9232-5d3b7d110998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:44.974033 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727a90c-4596-40d5-9232-5d3b7d110998 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:44.974471 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727a90c-4596-40d5-9232-5d3b7d110998 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:44.974750 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727a90c-4596-40d5-9232-5d3b7d110998 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:44.975557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727a90c-4596-40d5-9232-5d3b7d110998 None admin] RBAC: Adding query filter params (name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:44.975867 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727a90c-4596-40d5-9232-5d3b7d110998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:44.977283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727a90c-4596-40d5-9232-5d3b7d110998 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:44.990001 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 329/667] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:13:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:44.990001 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727a90c-4596-40d5-9232-5d3b7d110998 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:45.046817 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e9432-d820-48ef-a8c0-2419ecce3464 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:45.084134 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b73dc7-e390-4c8e-8b39-b7b5f93ee86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:45.084753 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b73dc7-e390-4c8e-8b39-b7b5f93ee86b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:45.085088 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b73dc7-e390-4c8e-8b39-b7b5f93ee86b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:45.085478 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b73dc7-e390-4c8e-8b39-b7b5f93ee86b None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:45.086231 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b73dc7-e390-4c8e-8b39-b7b5f93ee86b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:45.086558 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b73dc7-e390-4c8e-8b39-b7b5f93ee86b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:45.087935 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b73dc7-e390-4c8e-8b39-b7b5f93ee86b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:45.101165 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 339/668] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:45.102031 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b73dc7-e390-4c8e-8b39-b7b5f93ee86b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:45.111679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727a90c-4596-40d5-9232-5d3b7d110998 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:45.150158 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f587f-4ff9-4293-9ff9-9a46f25b2241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:45.150788 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f587f-4ff9-4293-9ff9-9a46f25b2241 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:45.151093 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f587f-4ff9-4293-9ff9-9a46f25b2241 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:45.151365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f587f-4ff9-4293-9ff9-9a46f25b2241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:45.164494 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f587f-4ff9-4293-9ff9-9a46f25b2241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:45.165987 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f587f-4ff9-4293-9ff9-9a46f25b2241 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:45.181237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-b81f587f-4ff9-4293-9ff9-9a46f25b2241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:45.181237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:45.181237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:45.181237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:45.181237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:45.181237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:45.181237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:45.181237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:45.181237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:45.181237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:45.181237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:45.181237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:45.181237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:45.181237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:45.181237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:45.181237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:45.182688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:45.182688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:45.182688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:45.182688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:45.182688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:45.182688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:45.182688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:45.182688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:45.182688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:45.182688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:45.182688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:45.182688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:45.182688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:45.182688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:45.182688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:45.182688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:45.182688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:45.184201 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:45.184201 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:45.184201 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:45.184201 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 330/669] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:45.184201 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f587f-4ff9-4293-9ff9-9a46f25b2241 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:45.195715 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:45.217010 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b73dc7-e390-4c8e-8b39-b7b5f93ee86b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:45.224672 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:45.256772 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0514231-32c9-4646-a4bf-33c502fd5c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:45.257392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0514231-32c9-4646-a4bf-33c502fd5c7b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:45.257667 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0514231-32c9-4646-a4bf-33c502fd5c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:45.257933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0514231-32c9-4646-a4bf-33c502fd5c7b None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:45.258576 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0514231-32c9-4646-a4bf-33c502fd5c7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:45.258872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0514231-32c9-4646-a4bf-33c502fd5c7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:45.260290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0514231-32c9-4646-a4bf-33c502fd5c7b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:45.272265 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 340/670] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:45.273103 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0514231-32c9-4646-a4bf-33c502fd5c7b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:45.306014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f587f-4ff9-4293-9ff9-9a46f25b2241 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:45.346701 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11755755-0fde-46a0-a76d-37557c4fc433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:45.347360 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11755755-0fde-46a0-a76d-37557c4fc433 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:45.347646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11755755-0fde-46a0-a76d-37557c4fc433 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:45.347916 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11755755-0fde-46a0-a76d-37557c4fc433 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:45.360936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11755755-0fde-46a0-a76d-37557c4fc433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:45.362434 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11755755-0fde-46a0-a76d-37557c4fc433 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:45.378922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-11755755-0fde-46a0-a76d-37557c4fc433 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:45.378922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:45.378922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:45.378922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:45.378922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:45.378922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:45.378922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:45.378922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:45.378922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:45.378922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:45.378922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:45.378922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:45.378922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:45.378922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:45.378922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:45.378922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:45.379977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:45.379977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:45.379977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:45.379977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:45.379977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:45.379977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:45.379977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:45.379977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:45.379977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:45.379977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:45.379977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:45.379977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:45.379977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:45.379977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:45.379977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:45.379977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:45.379977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:45.381285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:45.381285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:45.381285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:45.381285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 331/671] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:45.381510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11755755-0fde-46a0-a76d-37557c4fc433 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:45.476467 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:45.491484 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11755755-0fde-46a0-a76d-37557c4fc433 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:45.517002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0514231-32c9-4646-a4bf-33c502fd5c7b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:45.527949 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ebff0-8364-4ba2-a45b-81f3cb215187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:45.528604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ebff0-8364-4ba2-a45b-81f3cb215187 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:45.528889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ebff0-8364-4ba2-a45b-81f3cb215187 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:45.529183 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ebff0-8364-4ba2-a45b-81f3cb215187 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:45.529839 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ebff0-8364-4ba2-a45b-81f3cb215187 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:45.530159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ebff0-8364-4ba2-a45b-81f3cb215187 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:45.531536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839ebff0-8364-4ba2-a45b-81f3cb215187 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:45.544073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 332/672] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:45.544804 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ebff0-8364-4ba2-a45b-81f3cb215187 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:45.553824 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81076a-8648-4453-ab36-3cf054358d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:45.554463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81076a-8648-4453-ab36-3cf054358d9f None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:45.554743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81076a-8648-4453-ab36-3cf054358d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:45.555012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df81076a-8648-4453-ab36-3cf054358d9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:45.562214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df81076a-8648-4453-ab36-3cf054358d9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:45.563665 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df81076a-8648-4453-ab36-3cf054358d9f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:45.574548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-df81076a-8648-4453-ab36-3cf054358d9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:45.574548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:45.574548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:45.574548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:45.574548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:45.574548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:45.574548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:45.574548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:45.574548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:45.574548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:45.574548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:45.574548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:45.574548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:45.574548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:45.574548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:45.575628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:45.575628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:45.575628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:45.575628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:45.575628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:45.575628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:45.575628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:45.575628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:45.575628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:45.575628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:45.575628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:45.575628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:45.575628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:45.575628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:45.575628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:45.575628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:45.575628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:45.576763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:45.576763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:45.576763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:45.576763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:45.576763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:45.576763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:45.576763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:45.576763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:45.576763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 341/673] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:45.576763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81076a-8648-4453-ab36-3cf054358d9f None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:45.579750 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:45.644900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ebff0-8364-4ba2-a45b-81f3cb215187 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:45.680427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df81076a-8648-4453-ab36-3cf054358d9f None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:45.681683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555af799-40b2-4e48-8499-e67c58ebaf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:45.682983 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555af799-40b2-4e48-8499-e67c58ebaf35 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:45.683297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555af799-40b2-4e48-8499-e67c58ebaf35 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:45.683671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555af799-40b2-4e48-8499-e67c58ebaf35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:45.690907 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555af799-40b2-4e48-8499-e67c58ebaf35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:45.692397 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555af799-40b2-4e48-8499-e67c58ebaf35 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:45.703724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-555af799-40b2-4e48-8499-e67c58ebaf35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:45.703724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:45.703724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:45.703724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:45.703724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:45.703724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:45.703724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:45.703724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:45.703724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:45.703724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:45.703724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:45.703724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:45.703724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:45.703724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:45.703724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:45.704702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:45.704702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:45.704702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:45.704702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:45.704702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:45.704702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:45.704702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:45.704702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:45.704702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:45.704702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:45.704702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:45.704702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:45.704702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:45.704702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:45.704702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:45.704702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:45.704702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:45.705574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:45.705574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:45.705574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:45.705574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:45.705574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:45.705574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:45.705574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:45.705574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:45.705574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 333/674] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:45.706290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe157a8-e145-4b67-a6fd-06370579877e None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:45.706290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe157a8-e145-4b67-a6fd-06370579877e None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:45.706487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe157a8-e145-4b67-a6fd-06370579877e None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:45.707729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 334/675] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:45.708444 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe157a8-e145-4b67-a6fd-06370579877e None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:45.719567 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:45.722014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb511b1-6df8-4456-93dc-dc3164faf951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:45.722636 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb511b1-6df8-4456-93dc-dc3164faf951 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:45.722911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb511b1-6df8-4456-93dc-dc3164faf951 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:45.723177 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb511b1-6df8-4456-93dc-dc3164faf951 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:45.723830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb511b1-6df8-4456-93dc-dc3164faf951 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:45.724179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb511b1-6df8-4456-93dc-dc3164faf951 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:45.725574 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb511b1-6df8-4456-93dc-dc3164faf951 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:45.737041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 342/676] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:45.737793 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb511b1-6df8-4456-93dc-dc3164faf951 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:45.803078 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:45.812967 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe157a8-e145-4b67-a6fd-06370579877e None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:45.843657 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb511b1-6df8-4456-93dc-dc3164faf951 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:45.846662 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb64271-020e-4c29-97e5-76721909c4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:45.847287 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb64271-020e-4c29-97e5-76721909c4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:45.847562 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb64271-020e-4c29-97e5-76721909c4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:45.847827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb64271-020e-4c29-97e5-76721909c4a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:45.854617 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb64271-020e-4c29-97e5-76721909c4a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:45.856182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb64271-020e-4c29-97e5-76721909c4a0 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:45.878155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe0365-2022-41a4-b9bb-109257a3d753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:45.878792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe0365-2022-41a4-b9bb-109257a3d753 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:45.879069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe0365-2022-41a4-b9bb-109257a3d753 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:45.879338 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe0365-2022-41a4-b9bb-109257a3d753 None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:45.880077 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe0365-2022-41a4-b9bb-109257a3d753 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:45.880377 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe0365-2022-41a4-b9bb-109257a3d753 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:45.881878 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe0365-2022-41a4-b9bb-109257a3d753 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:45.895070 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 343/677] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:45.895973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe0365-2022-41a4-b9bb-109257a3d753 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:45.988014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-7cb64271-020e-4c29-97e5-76721909c4a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:45.988014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:45.988014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:45.988014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:45.988014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:45.988014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:45.988014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:45.988014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:45.988014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:45.988014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:45.988014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:45.988014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:45.988014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:45.988014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:45.988014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:45.988986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:45.988986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:45.988986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:45.988986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:45.988986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:45.988986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:45.988986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:45.988986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:45.988986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:45.988986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:45.988986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:45.988986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:45.988986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:45.988986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:45.988986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:45.988986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:45.988986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:45.989870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:45.989870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:45.989870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:45.989870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:45.989870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:45.989870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:45.989870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:45.989870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:45.992357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 335/678] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:45.993463 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ac680-3968-404e-a4f1-24377a5bfc3d None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:45.993769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ac680-3968-404e-a4f1-24377a5bfc3d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:45.994040 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ac680-3968-404e-a4f1-24377a5bfc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:46.003684 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe0365-2022-41a4-b9bb-109257a3d753 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:46.015214 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-351ac680-3968-404e-a4f1-24377a5bfc3d None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:46.016610 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-351ac680-3968-404e-a4f1-24377a5bfc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:46.027623 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:46.040704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4befd-a244-4450-b34b-02b9930a5b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:46.041331 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4befd-a244-4450-b34b-02b9930a5b7c None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:46.041612 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4befd-a244-4450-b34b-02b9930a5b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:46.041881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4befd-a244-4450-b34b-02b9930a5b7c None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:46.042583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4befd-a244-4450-b34b-02b9930a5b7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:46.042885 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4befd-a244-4450-b34b-02b9930a5b7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:46.044391 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd4befd-a244-4450-b34b-02b9930a5b7c None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:46.056915 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 344/679] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:46.058135 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4befd-a244-4450-b34b-02b9930a5b7c None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:46.080797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:46.100020 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 336/680] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:46.100020 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ac680-3968-404e-a4f1-24377a5bfc3d None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:46.165481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4befd-a244-4450-b34b-02b9930a5b7c None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:46.198512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ac680-3968-404e-a4f1-24377a5bfc3d None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:46.199838 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba5e09-07fe-4910-aa48-b5523a74f22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:46.200507 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba5e09-07fe-4910-aa48-b5523a74f22f None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:46.200780 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba5e09-07fe-4910-aa48-b5523a74f22f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:46.201055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba5e09-07fe-4910-aa48-b5523a74f22f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:46.208273 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba5e09-07fe-4910-aa48-b5523a74f22f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:46.209729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba5e09-07fe-4910-aa48-b5523a74f22f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:46.221458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-2dba5e09-07fe-4910-aa48-b5523a74f22f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 19:13:46.221458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:46.221458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:46.221458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:46.221458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:46.221458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:46.221458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:46.221458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:46.221458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:46.221458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:46.221458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:46.221458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:46.221458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:46.221458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:46.221458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:46.221458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:46.222446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:46.223405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:46.223405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:46.223405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:46.223405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:46.223405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:46.223405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:46.223405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:46.223405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:46.223405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 19:13:46.223405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:46.223405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 345/681] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:13:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:46.223405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba5e09-07fe-4910-aa48-b5523a74f22f None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:46.237501 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf00c88-6a1f-4466-8407-6f46fa1b9248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:46.238173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf00c88-6a1f-4466-8407-6f46fa1b9248 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:46.238402 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf00c88-6a1f-4466-8407-6f46fa1b9248 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:46.238673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf00c88-6a1f-4466-8407-6f46fa1b9248 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:46.239402 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf00c88-6a1f-4466-8407-6f46fa1b9248 None admin] RBAC: Adding query filter params (role.id=43aa4a9830054a09baa1f0aecfd125e2, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=f83e5f0b1d5e4f57bbf283d420f080ec) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:46.239709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf00c88-6a1f-4466-8407-6f46fa1b9248 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:46.241209 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf00c88-6a1f-4466-8407-6f46fa1b9248 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:46.247117 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 337/682] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:13:46 2022] GET /identity/v3/role_assignments?user.id=f83e5f0b1d5e4f57bbf283d420f080ec&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=43aa4a9830054a09baa1f0aecfd125e2 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:46.247932 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf00c88-6a1f-4466-8407-6f46fa1b9248 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:46.280600 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:46.333328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:46.356364 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba5e09-07fe-4910-aa48-b5523a74f22f None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:46.379043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf00c88-6a1f-4466-8407-6f46fa1b9248 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:46.394990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69379bf-2cc7-461b-9969-826ef2741ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:46.395613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69379bf-2cc7-461b-9969-826ef2741ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:46.395884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69379bf-2cc7-461b-9969-826ef2741ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:46.396204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69379bf-2cc7-461b-9969-826ef2741ba6 None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:46.396900 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69379bf-2cc7-461b-9969-826ef2741ba6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:46.397209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69379bf-2cc7-461b-9969-826ef2741ba6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:46.398678 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69379bf-2cc7-461b-9969-826ef2741ba6 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:46.413605 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 346/683] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:46.414470 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69379bf-2cc7-461b-9969-826ef2741ba6 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:46.427340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443e22d-d0c9-47dd-a748-b3a683109b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:46.428110 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443e22d-d0c9-47dd-a748-b3a683109b69 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:46.428356 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443e22d-d0c9-47dd-a748-b3a683109b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:46.428624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443e22d-d0c9-47dd-a748-b3a683109b69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:46.429356 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443e22d-d0c9-47dd-a748-b3a683109b69 None admin] RBAC: Adding query filter params (role.id=43aa4a9830054a09baa1f0aecfd125e2, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=a4946aec299f45bd8402cff3159d04d7) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:46.429659 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443e22d-d0c9-47dd-a748-b3a683109b69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:46.431231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443e22d-d0c9-47dd-a748-b3a683109b69 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:46.438058 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 338/684] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:13:46 2022] GET /identity/v3/role_assignments?user.id=a4946aec299f45bd8402cff3159d04d7&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=43aa4a9830054a09baa1f0aecfd125e2 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:46.438882 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443e22d-d0c9-47dd-a748-b3a683109b69 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:46.549281 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69379bf-2cc7-461b-9969-826ef2741ba6 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:46.551057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:46.574456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443e22d-d0c9-47dd-a748-b3a683109b69 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:46.590923 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd411c-c913-48d2-a589-5351e82d411b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:46.591721 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd411c-c913-48d2-a589-5351e82d411b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:46.592141 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd411c-c913-48d2-a589-5351e82d411b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:46.592508 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd411c-c913-48d2-a589-5351e82d411b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:46.604647 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd411c-c913-48d2-a589-5351e82d411b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:46.606324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd411c-c913-48d2-a589-5351e82d411b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:46.616681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:46.624837 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86fc35-6ba7-4451-8717-56ceb9e02602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:46.625253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-24bd411c-c913-48d2-a589-5351e82d411b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:46.625253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:46.625253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:46.625253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:46.625253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:46.625253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:46.625253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:46.625253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:46.625253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:46.625253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:46.625253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:46.625253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:46.625253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:46.625253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:46.625253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:46.625253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:46.626155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:46.626155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:46.626155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:46.626155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:46.626155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:46.626155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:46.626155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:46.626155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:46.626155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:46.626155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:46.626155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:46.626155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:46.626155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:46.626155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:46.626155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:46.626155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:46.626155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:46.627017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:46.627017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:46.627017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:46.627195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86fc35-6ba7-4451-8717-56ceb9e02602 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:46.627195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86fc35-6ba7-4451-8717-56ceb9e02602 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:46.627195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86fc35-6ba7-4451-8717-56ceb9e02602 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:46.627195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86fc35-6ba7-4451-8717-56ceb9e02602 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:46.627419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 347/685] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:46.627915 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86fc35-6ba7-4451-8717-56ceb9e02602 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:46.628951 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd411c-c913-48d2-a589-5351e82d411b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:46.629206 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d86fc35-6ba7-4451-8717-56ceb9e02602 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:46.655729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 339/686] 10.222.0.17 () {56 vars in 1135 bytes} [Sun Aug 7 19:13:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:46.656750 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86fc35-6ba7-4451-8717-56ceb9e02602 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:46.756650 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd411c-c913-48d2-a589-5351e82d411b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:46.782867 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d86fc35-6ba7-4451-8717-56ceb9e02602 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:46.790937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cfc84-f24e-4d42-99c4-806427f865a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:46.791573 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cfc84-f24e-4d42-99c4-806427f865a2 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:46.791850 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cfc84-f24e-4d42-99c4-806427f865a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:46.792162 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cfc84-f24e-4d42-99c4-806427f865a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:46.792939 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607cfc84-f24e-4d42-99c4-806427f865a2 None admin] RBAC: Adding query filter params (role.id=43aa4a9830054a09baa1f0aecfd125e2, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=08229e9400b14e8a9cc8e8c32dc48285) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:46.793242 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607cfc84-f24e-4d42-99c4-806427f865a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:46.794685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607cfc84-f24e-4d42-99c4-806427f865a2 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:46.804168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 348/687] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:13:46 2022] GET /identity/v3/role_assignments?user.id=08229e9400b14e8a9cc8e8c32dc48285&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=43aa4a9830054a09baa1f0aecfd125e2 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:46.804168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cfc84-f24e-4d42-99c4-806427f865a2 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:46.833494 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461417c-1565-4542-97d6-a711d546f2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:46.834123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461417c-1565-4542-97d6-a711d546f2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:46.834402 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461417c-1565-4542-97d6-a711d546f2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:46.834675 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461417c-1565-4542-97d6-a711d546f2e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:46.835349 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461417c-1565-4542-97d6-a711d546f2e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:46.835655 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461417c-1565-4542-97d6-a711d546f2e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:46.837159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461417c-1565-4542-97d6-a711d546f2e3 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:46.851035 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 340/688] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:46.852892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:46.856800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461417c-1565-4542-97d6-a711d546f2e3 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:46.946505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:47.002352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cfc84-f24e-4d42-99c4-806427f865a2 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:47.035220 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461417c-1565-4542-97d6-a711d546f2e3 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:47.058136 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0ed72-b888-4be5-b0ab-881bed76f05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:47.058683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0ed72-b888-4be5-b0ab-881bed76f05f None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:47.058963 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0ed72-b888-4be5-b0ab-881bed76f05f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:47.059231 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0ed72-b888-4be5-b0ab-881bed76f05f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:47.080706 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07d777-07b5-4488-9d0b-e9a2b6db2710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:47.081718 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07d777-07b5-4488-9d0b-e9a2b6db2710 None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:47.082157 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07d777-07b5-4488-9d0b-e9a2b6db2710 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:47.082576 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07d777-07b5-4488-9d0b-e9a2b6db2710 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:47.083647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07d777-07b5-4488-9d0b-e9a2b6db2710 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:47.085764 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07d777-07b5-4488-9d0b-e9a2b6db2710 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:47.119890 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 341/689] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:13:46 2022] POST /identity/v3/users => generated 254 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:13:47.149041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0ed72-b888-4be5-b0ab-881bed76f05f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:47.150386 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0ed72-b888-4be5-b0ab-881bed76f05f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:47.171759 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:47.268985 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 349/690] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:47.274385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07d777-07b5-4488-9d0b-e9a2b6db2710 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:47.411489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:47.414781 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07d777-07b5-4488-9d0b-e9a2b6db2710 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:47.459367 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20767d56-7165-49d1-80fd-1065701c6911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:47.460022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20767d56-7165-49d1-80fd-1065701c6911 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:47.460329 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20767d56-7165-49d1-80fd-1065701c6911 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:47.460599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20767d56-7165-49d1-80fd-1065701c6911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:47.471584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20767d56-7165-49d1-80fd-1065701c6911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:47.473109 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20767d56-7165-49d1-80fd-1065701c6911 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:47.490431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-20767d56-7165-49d1-80fd-1065701c6911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:47.490431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:47.490431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:47.490431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:47.490431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:47.490431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:47.490431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:47.490431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:47.490431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:47.490431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:47.490431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:47.490431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:47.490431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:47.490431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:47.490431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:47.490431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:47.491453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:47.491453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:47.491453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:47.491453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:47.491453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:47.491453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:47.491453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:47.491453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:47.491453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:47.491453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:47.491453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:47.491453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:47.491453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:47.491453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:47.491453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:47.491453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:47.491453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:47.492759 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:47.492759 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:47.492759 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:47.492759 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 342/691] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:47.499417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0ed72-b888-4be5-b0ab-881bed76f05f None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:47.621276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0ed72-b888-4be5-b0ab-881bed76f05f None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:47.640462 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:47.666819 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ecc996-8115-42b9-8efb-a331f4408593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:47.667894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecc996-8115-42b9-8efb-a331f4408593 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:47.668401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecc996-8115-42b9-8efb-a331f4408593 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:47.668878 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecc996-8115-42b9-8efb-a331f4408593 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:47.669722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ecc996-8115-42b9-8efb-a331f4408593 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:47.670218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ecc996-8115-42b9-8efb-a331f4408593 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:47.671822 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ecc996-8115-42b9-8efb-a331f4408593 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:47.689927 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 350/692] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:47.697197 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20767d56-7165-49d1-80fd-1065701c6911 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:47.843779 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20767d56-7165-49d1-80fd-1065701c6911 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:47.890534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51aa394-e9eb-404b-a780-75dcc1cfe4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:47.891177 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aa394-e9eb-404b-a780-75dcc1cfe4f1 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:47.891451 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aa394-e9eb-404b-a780-75dcc1cfe4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:47.891726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aa394-e9eb-404b-a780-75dcc1cfe4f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:47.901395 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51aa394-e9eb-404b-a780-75dcc1cfe4f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:47.902898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51aa394-e9eb-404b-a780-75dcc1cfe4f1 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:47.914145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-f51aa394-e9eb-404b-a780-75dcc1cfe4f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:47.914145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:47.914145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:47.914145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:47.914145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:47.914145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:47.914145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:47.914145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:47.914145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:47.914145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:47.914145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:47.914145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:47.914145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:47.914145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:47.914145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:47.915016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:47.915016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:47.915016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:47.915016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:47.915016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:47.915016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:47.915016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:47.915016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:47.915016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:47.915016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:47.915016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:47.915016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:47.915016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:47.915016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:47.915016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:47.915016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:47.915016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:47.916424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:47.916424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:47.916424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:47.916424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:47.916424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:47.916424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:47.916424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:47.916424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:47.916424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 343/693] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:47.921312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ecc996-8115-42b9-8efb-a331f4408593 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:48.089851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ecc996-8115-42b9-8efb-a331f4408593 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:48.124375 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70cd75c-da41-4433-beea-821c9d903984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:48.125004 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cd75c-da41-4433-beea-821c9d903984 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:48.125275 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cd75c-da41-4433-beea-821c9d903984 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:48.125548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70cd75c-da41-4433-beea-821c9d903984 None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:48.126244 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cd75c-da41-4433-beea-821c9d903984 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:48.126547 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cd75c-da41-4433-beea-821c9d903984 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:48.128010 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70cd75c-da41-4433-beea-821c9d903984 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:48.141403 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 351/694] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:48.149121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51aa394-e9eb-404b-a780-75dcc1cfe4f1 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:48.172304 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:48.290109 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51aa394-e9eb-404b-a780-75dcc1cfe4f1 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:48.331492 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b065850-856a-400f-bd15-2e1854131c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:48.332453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b065850-856a-400f-bd15-2e1854131c75 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:48.332875 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b065850-856a-400f-bd15-2e1854131c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:48.333325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b065850-856a-400f-bd15-2e1854131c75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:48.334198 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b065850-856a-400f-bd15-2e1854131c75 None admin] RBAC: Adding query filter params (role.id=43aa4a9830054a09baa1f0aecfd125e2, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=db51971158a946c98a8ed716ecc5d72b) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:48.334666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b065850-856a-400f-bd15-2e1854131c75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:48.336269 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b065850-856a-400f-bd15-2e1854131c75 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:48.343108 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 344/695] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:13:48 2022] GET /identity/v3/role_assignments?user.id=db51971158a946c98a8ed716ecc5d72b&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=43aa4a9830054a09baa1f0aecfd125e2 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:48.655834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c00af-1a78-4f05-aa5c-d4526f0b9c2e None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:48.656372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c00af-1a78-4f05-aa5c-d4526f0b9c2e None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:48.656372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c00af-1a78-4f05-aa5c-d4526f0b9c2e None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:48.657881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 352/696] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:48.663377 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698181d-1f40-4eff-bb56-f8ca3dd42e1f None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:48.663663 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698181d-1f40-4eff-bb56-f8ca3dd42e1f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:48.663936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698181d-1f40-4eff-bb56-f8ca3dd42e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:48.683531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-e698181d-1f40-4eff-bb56-f8ca3dd42e1f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:48.684744 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e698181d-1f40-4eff-bb56-f8ca3dd42e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:48.760509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 345/697] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 19:13:48.781934 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c00af-1a78-4f05-aa5c-d4526f0b9c2e None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:48.832233 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:48.894479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c00af-1a78-4f05-aa5c-d4526f0b9c2e None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:48.933475 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753e7d6-af55-406b-9a06-4967d6317495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:48.934151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753e7d6-af55-406b-9a06-4967d6317495 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:48.934434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753e7d6-af55-406b-9a06-4967d6317495 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:48.934710 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753e7d6-af55-406b-9a06-4967d6317495 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:48.935332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753e7d6-af55-406b-9a06-4967d6317495 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:48.936787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753e7d6-af55-406b-9a06-4967d6317495 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:48.948894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-0753e7d6-af55-406b-9a06-4967d6317495 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 19:13:48.948894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:48.948894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:13:48.948894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:13:48.948894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:13:48.948894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:13:48.948894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:13:48.948894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:13:48.948894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 19:13:48.948894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:48.948894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:13:48.948894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:48.948894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:48.948894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:48.948894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:48.948894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:48.948894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:48.949995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:48.949995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:48.949995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:48.949995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:48.949995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:48.949995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:48.949995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:13:48.949995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:13:48.949995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:13:48.949995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:13:48.949995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:48.949995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:48.949995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:48.949995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:48.949995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:48.949995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:48.950979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:48.950979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:48.950979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:48.950979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:48.950979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:48.950979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:48.950979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:48.950979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:48.950979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:48.950979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:48.950979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:13:48.950979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:13:48.950979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 19:13:48.950979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:48.950979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 353/698] 10.222.0.17 () {56 vars in 1129 bytes} [Sun Aug 7 19:13:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:48.956273 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698181d-1f40-4eff-bb56-f8ca3dd42e1f None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:48.968357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:49.091705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698181d-1f40-4eff-bb56-f8ca3dd42e1f None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:49.092119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af689947-8917-4414-810c-819e48d410e6 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:49.092119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af689947-8917-4414-810c-819e48d410e6 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:49.092119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af689947-8917-4414-810c-819e48d410e6 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:49.092119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 354/699] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:13:49.097178 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5bd97-1be8-4e24-ae0f-62c726d3e5c3 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:49.097451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5bd97-1be8-4e24-ae0f-62c726d3e5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:49.097712 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5bd97-1be8-4e24-ae0f-62c726d3e5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:49.132567 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb943f6d-dd42-40de-9a92-8bfde723ae01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:49.133610 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb943f6d-dd42-40de-9a92-8bfde723ae01 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:49.134126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb943f6d-dd42-40de-9a92-8bfde723ae01 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:49.134590 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb943f6d-dd42-40de-9a92-8bfde723ae01 None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:49.135418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb943f6d-dd42-40de-9a92-8bfde723ae01 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:49.135915 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb943f6d-dd42-40de-9a92-8bfde723ae01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:49.137570 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb943f6d-dd42-40de-9a92-8bfde723ae01 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:49.145306 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-e4e5bd97-1be8-4e24-ae0f-62c726d3e5c3 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:49.148768 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e4e5bd97-1be8-4e24-ae0f-62c726d3e5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:49.151543 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 346/700] 10.222.0.17 () {56 vars in 1124 bytes} [Sun Aug 7 19:13:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:49.154607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb943f6d-dd42-40de-9a92-8bfde723ae01 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:49.238408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:49.272683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb943f6d-dd42-40de-9a92-8bfde723ae01 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:49.309614 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87278388-3415-40f8-b099-5c14a9368b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:49.311132 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87278388-3415-40f8-b099-5c14a9368b5d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:49.311606 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87278388-3415-40f8-b099-5c14a9368b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:49.312142 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87278388-3415-40f8-b099-5c14a9368b5d None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:49.312971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87278388-3415-40f8-b099-5c14a9368b5d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:49.313471 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87278388-3415-40f8-b099-5c14a9368b5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:49.315061 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87278388-3415-40f8-b099-5c14a9368b5d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:49.322285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 347/701] 10.222.0.17 () {56 vars in 1124 bytes} [Sun Aug 7 19:13:49 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:49.347678 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 355/702] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:49.365681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e5b17-79ba-4e1e-be46-0e267175485d None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:49.366313 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e5b17-79ba-4e1e-be46-0e267175485d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:49.366788 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e5b17-79ba-4e1e-be46-0e267175485d None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:49.368722 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 348/703] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:49.374922 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d3c86-eb12-47eb-b4dd-8659f56f26bf None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:49.375576 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d3c86-eb12-47eb-b4dd-8659f56f26bf None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:49.376094 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d3c86-eb12-47eb-b4dd-8659f56f26bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:49.377632 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e5b17-79ba-4e1e-be46-0e267175485d None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:49.411688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-bc7d3c86-eb12-47eb-b4dd-8659f56f26bf None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:49.411688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-bc7d3c86-eb12-47eb-b4dd-8659f56f26bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:49.476384 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:49.526956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 356/704] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:49.550336 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d3c86-eb12-47eb-b4dd-8659f56f26bf None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:49.551782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e5b17-79ba-4e1e-be46-0e267175485d None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:49.602874 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca6f31-3906-4b0d-ab38-c0cbab1045b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:49.603519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca6f31-3906-4b0d-ab38-c0cbab1045b1 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:49.603795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca6f31-3906-4b0d-ab38-c0cbab1045b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:49.604093 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca6f31-3906-4b0d-ab38-c0cbab1045b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:49.609240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:49.626019 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca6f31-3906-4b0d-ab38-c0cbab1045b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:49.635583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-16ca6f31-3906-4b0d-ab38-c0cbab1045b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:13:49.635583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:49.635583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:49.635583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:49.635583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:49.635583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:49.635583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:49.635583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:49.635583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:49.635583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:49.635583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:49.635583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:49.635583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:49.635583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:49.635583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:49.635583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:49.636646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:49.636646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:49.636646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:49.636646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:49.636646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:49.636646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:49.636646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:49.636646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:49.636646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:49.636646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:49.636646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:49.636646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:49.636646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:49.636646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:49.636646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:49.636646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:49.637468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:49.637468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:49.637468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:49.637468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:49.637468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:49.637468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:49.637468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:49.637468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:49.637468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:49.637468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:49.637468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:13:49.637468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:49.640640 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 349/705] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:13:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:49.642882 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca6f31-3906-4b0d-ab38-c0cbab1045b1 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:49.718065 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d3c86-eb12-47eb-b4dd-8659f56f26bf None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:49.780133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca6f31-3906-4b0d-ab38-c0cbab1045b1 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:49.792523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b547908-4837-431a-b073-3dbf89fad5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:49.792970 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b547908-4837-431a-b073-3dbf89fad5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:49.793248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b547908-4837-431a-b073-3dbf89fad5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:49.793520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b547908-4837-431a-b073-3dbf89fad5c0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:49.794117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b547908-4837-431a-b073-3dbf89fad5c0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:49.795502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b547908-4837-431a-b073-3dbf89fad5c0 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:49.808003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-6b547908-4837-431a-b073-3dbf89fad5c0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 19:13:49.808003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:49.808003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:13:49.808003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:13:49.808003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:13:49.808003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:13:49.808003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:13:49.808003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:13:49.808003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 19:13:49.808003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:49.808003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:13:49.808003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:49.808003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:49.808003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:49.808003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:49.808003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:49.808003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:49.809179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:49.809179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:49.809179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:49.809179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:49.809179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:49.809179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:49.809179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:13:49.809179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:13:49.809179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:13:49.809179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:13:49.809179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:49.809179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:49.809179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:49.809179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:49.809179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:49.809179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:49.810040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:49.810040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:49.810040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:49.810040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:49.810040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:49.810040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:49.810040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:49.810040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:49.810040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:49.810040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:49.810040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:13:49.810040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:13:49.810040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 19:13:49.810040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:49.815402 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 357/706] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:49 2022] GET /identity/v3/services/network => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:49.816725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b547908-4837-431a-b073-3dbf89fad5c0 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:49.830013 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f9bf97-21d1-49a2-8566-ec83dbfc63b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:49.830963 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9bf97-21d1-49a2-8566-ec83dbfc63b5 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:49.831482 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9bf97-21d1-49a2-8566-ec83dbfc63b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:49.831954 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f9bf97-21d1-49a2-8566-ec83dbfc63b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:49.832798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9bf97-21d1-49a2-8566-ec83dbfc63b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:49.833149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9bf97-21d1-49a2-8566-ec83dbfc63b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:49.834647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f9bf97-21d1-49a2-8566-ec83dbfc63b5 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:49.844061 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 350/707] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:49.845383 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364936f4-7cfa-4180-95bb-c5b229676ec9 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:49.845770 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364936f4-7cfa-4180-95bb-c5b229676ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:49.846104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364936f4-7cfa-4180-95bb-c5b229676ec9 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:49.848027 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 351/708] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:49.849170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364936f4-7cfa-4180-95bb-c5b229676ec9 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:49.854506 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:49.913684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:49.940608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b547908-4837-431a-b073-3dbf89fad5c0 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:49.975629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364936f4-7cfa-4180-95bb-c5b229676ec9 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:49.976557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab4bcc-18b2-4ac5-9b1e-1b2b7cfcf3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:49.977167 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab4bcc-18b2-4ac5-9b1e-1b2b7cfcf3af None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:49.977441 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab4bcc-18b2-4ac5-9b1e-1b2b7cfcf3af None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:49.977712 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab4bcc-18b2-4ac5-9b1e-1b2b7cfcf3af None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:49.978408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab4bcc-18b2-4ac5-9b1e-1b2b7cfcf3af None admin] RBAC: Adding query filter params (name=network) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:49.978628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab4bcc-18b2-4ac5-9b1e-1b2b7cfcf3af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:49.980452 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab4bcc-18b2-4ac5-9b1e-1b2b7cfcf3af None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:49.985498 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 358/709] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Aug 7 19:13:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:49.987539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293e90a-e3f4-436b-bfd1-6e06aa41bb71 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:49.987980 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293e90a-e3f4-436b-bfd1-6e06aa41bb71 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:49.988309 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293e90a-e3f4-436b-bfd1-6e06aa41bb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:50.007711 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-1293e90a-e3f4-436b-bfd1-6e06aa41bb71 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:50.009134 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-1293e90a-e3f4-436b-bfd1-6e06aa41bb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:50.019260 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd772e8a-fd59-4327-ad3c-6851a64e4f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:50.020611 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd772e8a-fd59-4327-ad3c-6851a64e4f18 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:50.021441 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd772e8a-fd59-4327-ad3c-6851a64e4f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:50.021919 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd772e8a-fd59-4327-ad3c-6851a64e4f18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:50.036309 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd772e8a-fd59-4327-ad3c-6851a64e4f18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:50.038926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd772e8a-fd59-4327-ad3c-6851a64e4f18 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:50.057586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-dd772e8a-fd59-4327-ad3c-6851a64e4f18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:50.057586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:50.057586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:50.057586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:50.057586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:50.057586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:50.057586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:50.057586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:50.057586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:50.057586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:50.057586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:50.057586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:50.057586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:50.057586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:50.057586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:50.057586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:50.058641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:50.058641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:50.058641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:50.058641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:50.058641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:50.058641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:50.058641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:50.058641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:50.058641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:50.058641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:50.058641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:50.058641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:50.058641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:50.058641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:50.058641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:50.058641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:50.058641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:50.059511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:50.059511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:50.059511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:50.059511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 352/710] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:50.060612 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd772e8a-fd59-4327-ad3c-6851a64e4f18 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:50.096418 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 359/711] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:50.097751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293e90a-e3f4-436b-bfd1-6e06aa41bb71 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:50.121924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:50.190498 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd772e8a-fd59-4327-ad3c-6851a64e4f18 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:50.200981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:50.226587 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293e90a-e3f4-436b-bfd1-6e06aa41bb71 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:50.227641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd021e39-8db2-41c4-b5b5-7c7764edd2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:50.228372 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd021e39-8db2-41c4-b5b5-7c7764edd2ed None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:50.228770 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd021e39-8db2-41c4-b5b5-7c7764edd2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:50.229098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd021e39-8db2-41c4-b5b5-7c7764edd2ed None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:50.229817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd021e39-8db2-41c4-b5b5-7c7764edd2ed None admin] RBAC: Adding query filter params (type=network) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:50.230919 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd021e39-8db2-41c4-b5b5-7c7764edd2ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:50.231840 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd021e39-8db2-41c4-b5b5-7c7764edd2ed None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:50.237162 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 353/712] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Aug 7 19:13:50 2022] GET /identity/v3/services?type=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:50.237965 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd021e39-8db2-41c4-b5b5-7c7764edd2ed None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:50.274386 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7d63c-3621-4909-849e-a23eb35018ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:50.275271 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7d63c-3621-4909-849e-a23eb35018ac None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:50.275922 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7d63c-3621-4909-849e-a23eb35018ac None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:50.276382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7d63c-3621-4909-849e-a23eb35018ac None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:50.278161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7d63c-3621-4909-849e-a23eb35018ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:50.278616 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7d63c-3621-4909-849e-a23eb35018ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:50.280188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7d63c-3621-4909-849e-a23eb35018ac None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:50.296899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 360/713] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:50.302315 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7d63c-3621-4909-849e-a23eb35018ac None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:50.348917 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd021e39-8db2-41c4-b5b5-7c7764edd2ed None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:50.386044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40fcd2f-1e46-4ae9-a743-70ea6a50a752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:50.386913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40fcd2f-1e46-4ae9-a743-70ea6a50a752 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:50.387281 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40fcd2f-1e46-4ae9-a743-70ea6a50a752 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:50.387670 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40fcd2f-1e46-4ae9-a743-70ea6a50a752 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:50.397402 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:50.400658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40fcd2f-1e46-4ae9-a743-70ea6a50a752 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:50.407250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-c40fcd2f-1e46-4ae9-a743-70ea6a50a752 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:50.407250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:50.407250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:50.407250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:50.407250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:50.407250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:50.407250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:50.407250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:50.407250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:50.407250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:50.407250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:50.407250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:50.407250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:50.407250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:50.407250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:50.407250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:50.408357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:50.408357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:50.408357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:50.408357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:50.408357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:50.408357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:50.408357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:50.408357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:50.408357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:50.408357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:50.408357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:50.408357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:50.408357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:50.408357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:50.408357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:50.408357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:50.409213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:50.409213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:50.409213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:50.409213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:50.409213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:50.409213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:50.409213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:50.409213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:50.409213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:50.409213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:50.409213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:50.409213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:50.409213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 354/714] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:50.414988 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40fcd2f-1e46-4ae9-a743-70ea6a50a752 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:50.424732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7d63c-3621-4909-849e-a23eb35018ac None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:50.449748 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:50.463833 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b37fbc-1045-4398-8403-f03f9195e373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:50.465018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b37fbc-1045-4398-8403-f03f9195e373 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:50.465466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b37fbc-1045-4398-8403-f03f9195e373 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:50.465901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b37fbc-1045-4398-8403-f03f9195e373 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:50.474365 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b37fbc-1045-4398-8403-f03f9195e373 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:50.476156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b37fbc-1045-4398-8403-f03f9195e373 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:50.488729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-43b37fbc-1045-4398-8403-f03f9195e373 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:13:50.488729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:50.488729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:50.488729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:50.488729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:50.488729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:50.488729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:50.488729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:50.488729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:50.488729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:50.488729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:50.488729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:50.488729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:50.488729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:50.488729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:50.488729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:50.489839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:50.490807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:50.490807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:50.490807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:50.490807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:50.490807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:50.490807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:50.490807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:50.490807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:50.490807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:13:50.490807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:50.493358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 361/715] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Aug 7 19:13:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:50.501603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b37fbc-1045-4398-8403-f03f9195e373 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:50.533217 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40fcd2f-1e46-4ae9-a743-70ea6a50a752 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:50.581272 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05523e-1b7d-4b73-8069-d22ff7e47eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:50.581804 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05523e-1b7d-4b73-8069-d22ff7e47eff None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:50.582079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05523e-1b7d-4b73-8069-d22ff7e47eff None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:50.582349 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05523e-1b7d-4b73-8069-d22ff7e47eff None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:50.583018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05523e-1b7d-4b73-8069-d22ff7e47eff None admin] RBAC: Adding query filter params (name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:50.583311 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05523e-1b7d-4b73-8069-d22ff7e47eff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:50.584742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05523e-1b7d-4b73-8069-d22ff7e47eff None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:50.593685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 355/716] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:13:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:50.599184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05523e-1b7d-4b73-8069-d22ff7e47eff None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:50.630490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b37fbc-1045-4398-8403-f03f9195e373 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:50.668694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc507ebf-0115-41e7-9914-e3716eb4725d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:50.669736 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc507ebf-0115-41e7-9914-e3716eb4725d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:50.670294 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc507ebf-0115-41e7-9914-e3716eb4725d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:50.670781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc507ebf-0115-41e7-9914-e3716eb4725d None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:50.671818 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc507ebf-0115-41e7-9914-e3716eb4725d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:50.672341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc507ebf-0115-41e7-9914-e3716eb4725d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:50.673996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc507ebf-0115-41e7-9914-e3716eb4725d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:50.685407 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:50.696146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 362/717] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Aug 7 19:13:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:50.699449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:50.701331 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc507ebf-0115-41e7-9914-e3716eb4725d None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:50.716083 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05523e-1b7d-4b73-8069-d22ff7e47eff None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:50.824734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5a9e8-deb4-4f16-85f7-54f92f1bc55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:50.825396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5a9e8-deb4-4f16-85f7-54f92f1bc55d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:50.825679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5a9e8-deb4-4f16-85f7-54f92f1bc55d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:50.825952 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5a9e8-deb4-4f16-85f7-54f92f1bc55d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:50.847459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc507ebf-0115-41e7-9914-e3716eb4725d None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:50.849955 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5a9e8-deb4-4f16-85f7-54f92f1bc55d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:50.851425 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5a9e8-deb4-4f16-85f7-54f92f1bc55d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:50.881388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-68e5a9e8-deb4-4f16-85f7-54f92f1bc55d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:50.881388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:50.881388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:50.881388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:50.881388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:50.881388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:50.881388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:50.881388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:50.881388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:50.881388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:50.881388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:50.881388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:50.881388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:50.881388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:50.881388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:50.881388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:50.882382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:50.882382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:50.882382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:50.882382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:50.882382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:50.882382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:50.882382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:50.882382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:50.882382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:50.882382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:50.882382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:50.882382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:50.882382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:50.882382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:50.882382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:50.882382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:50.882382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:50.883257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:50.883257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:50.883257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:50.883257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 356/718] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:50.888331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5a9e8-deb4-4f16-85f7-54f92f1bc55d None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:50.893394 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5c739-8775-4f27-ac23-8b3f7485dc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:50.894029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5c739-8775-4f27-ac23-8b3f7485dc11 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:50.894303 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5c739-8775-4f27-ac23-8b3f7485dc11 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:50.894572 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5c739-8775-4f27-ac23-8b3f7485dc11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:51.036165 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5a9e8-deb4-4f16-85f7-54f92f1bc55d None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:51.063573 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:51.073920 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5c739-8775-4f27-ac23-8b3f7485dc11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:51.077126 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d5c739-8775-4f27-ac23-8b3f7485dc11 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:51.080351 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35250b16-6bca-44e4-ad8f-5e1e806e8c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:51.081207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35250b16-6bca-44e4-ad8f-5e1e806e8c38 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:51.081621 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35250b16-6bca-44e4-ad8f-5e1e806e8c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:51.082055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35250b16-6bca-44e4-ad8f-5e1e806e8c38 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:51.082837 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35250b16-6bca-44e4-ad8f-5e1e806e8c38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:51.084427 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35250b16-6bca-44e4-ad8f-5e1e806e8c38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:51.085934 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35250b16-6bca-44e4-ad8f-5e1e806e8c38 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:51.103865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 357/719] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:51.105376 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7b509-5594-4ffc-8c68-a1df56a98680 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:51.105762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7b509-5594-4ffc-8c68-a1df56a98680 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:51.106136 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7b509-5594-4ffc-8c68-a1df56a98680 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:51.107733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 358/720] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:51.112608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7b509-5594-4ffc-8c68-a1df56a98680 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:51.122155 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:51.212176 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 363/721] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 19:13:51.213759 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b4761-7a8c-41f3-810f-c0aa453000bd None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:51.214167 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b4761-7a8c-41f3-810f-c0aa453000bd None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:51.214635 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b4761-7a8c-41f3-810f-c0aa453000bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:51.223027 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7b509-5594-4ffc-8c68-a1df56a98680 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:51.237188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-a44b4761-7a8c-41f3-810f-c0aa453000bd None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:51.239285 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a44b4761-7a8c-41f3-810f-c0aa453000bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:51.258300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c102c-8d31-4cf7-8ec1-bd12d66a4fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:51.259152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c102c-8d31-4cf7-8ec1-bd12d66a4fda None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:51.259489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c102c-8d31-4cf7-8ec1-bd12d66a4fda None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:51.259849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c102c-8d31-4cf7-8ec1-bd12d66a4fda None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:51.267701 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c102c-8d31-4cf7-8ec1-bd12d66a4fda None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:51.269322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c102c-8d31-4cf7-8ec1-bd12d66a4fda None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:51.281556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-7c3c102c-8d31-4cf7-8ec1-bd12d66a4fda None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 19:13:51.281556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:51.281556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:51.281556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:51.281556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:51.281556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:51.281556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:51.281556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:51.281556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:51.281556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:51.281556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:51.281556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:51.281556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:51.281556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:51.281556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:51.281556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:51.282741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:51.283889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:51.283889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:51.283889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:51.283889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:51.283889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:51.283889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:51.283889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:51.283889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:51.283889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 19:13:51.283889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:51.290104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 359/722] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Aug 7 19:13:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:51.291097 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c102c-8d31-4cf7-8ec1-bd12d66a4fda None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:51.326506 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 364/723] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:51.327596 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b4761-7a8c-41f3-810f-c0aa453000bd None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:51.366583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:51.388538 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:51.426433 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c102c-8d31-4cf7-8ec1-bd12d66a4fda None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:51.461208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b4761-7a8c-41f3-810f-c0aa453000bd None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:51.462460 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a00c3-b749-47d1-a689-c63838cf92e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:51.463338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a00c3-b749-47d1-a689-c63838cf92e9 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:51.463704 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a00c3-b749-47d1-a689-c63838cf92e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:51.464112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a00c3-b749-47d1-a689-c63838cf92e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:51.475138 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a00c3-b749-47d1-a689-c63838cf92e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:51.476828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a00c3-b749-47d1-a689-c63838cf92e9 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:51.491192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-2d9a00c3-b749-47d1-a689-c63838cf92e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:51.491192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:51.491192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:51.491192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:51.491192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:51.491192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:51.491192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:51.491192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:51.491192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:51.491192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:51.491192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:51.491192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:51.491192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:51.491192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:51.491192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:51.491192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:51.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:51.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:51.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:51.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:51.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:51.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:51.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:51.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:51.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:51.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:51.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:51.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:51.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:51.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:51.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:51.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:51.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:51.493516 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:51.493516 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:51.493516 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:51.499372 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 360/724] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:51.500231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a00c3-b749-47d1-a689-c63838cf92e9 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:51.505486 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ce175-9c1f-4c15-afdd-5c2b57422874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:51.506297 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ce175-9c1f-4c15-afdd-5c2b57422874 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:51.506746 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ce175-9c1f-4c15-afdd-5c2b57422874 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:51.507162 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ce175-9c1f-4c15-afdd-5c2b57422874 None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:51.508093 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ce175-9c1f-4c15-afdd-5c2b57422874 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:51.508559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ce175-9c1f-4c15-afdd-5c2b57422874 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:51.510708 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ce175-9c1f-4c15-afdd-5c2b57422874 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:51.538709 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 365/725] 10.222.0.17 () {56 vars in 1149 bytes} [Sun Aug 7 19:13:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:51.539812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ce175-9c1f-4c15-afdd-5c2b57422874 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:51.629427 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a00c3-b749-47d1-a689-c63838cf92e9 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:51.635255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:51.649368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:51.655026 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ce175-9c1f-4c15-afdd-5c2b57422874 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:51.669131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbff1b2-9300-4a37-a8e4-acedfe2960ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:51.669807 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbff1b2-9300-4a37-a8e4-acedfe2960ce None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:51.670054 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbff1b2-9300-4a37-a8e4-acedfe2960ce None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:51.670323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbff1b2-9300-4a37-a8e4-acedfe2960ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:51.681364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbff1b2-9300-4a37-a8e4-acedfe2960ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:51.687569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-cfbff1b2-9300-4a37-a8e4-acedfe2960ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:13:51.687569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:51.687569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:51.687569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:51.687569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:51.687569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:51.687569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:51.687569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:51.687569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:51.687569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:51.687569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:51.687569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:51.687569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:51.687569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:51.687569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:51.687569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:51.688858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:51.688858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:51.688858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:51.688858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:51.688858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:51.688858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:51.688858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:51.688858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:51.688858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:51.688858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:51.688858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:51.688858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:51.688858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:51.688858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:51.688858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:51.688858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:51.690016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:51.690016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:51.690016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:51.690016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:51.690016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:51.690016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:51.690016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:51.690016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:51.690016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:51.690016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:51.690016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:13:51.690016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:51.690016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 361/726] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:13:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:51.691017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbff1b2-9300-4a37-a8e4-acedfe2960ce None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:51.700706 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924279d-fb80-4754-884c-3c567095a946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:51.701345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924279d-fb80-4754-884c-3c567095a946 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:51.701624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924279d-fb80-4754-884c-3c567095a946 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:51.701896 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924279d-fb80-4754-884c-3c567095a946 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:51.702550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924279d-fb80-4754-884c-3c567095a946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:51.702852 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924279d-fb80-4754-884c-3c567095a946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:51.704291 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924279d-fb80-4754-884c-3c567095a946 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:51.720891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 366/727] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:51.720891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ce97f-7a2f-4227-b9ed-789cbee52a87 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:51.720891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ce97f-7a2f-4227-b9ed-789cbee52a87 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:51.720891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ce97f-7a2f-4227-b9ed-789cbee52a87 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:51.720891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 367/728] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:51.720891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ce97f-7a2f-4227-b9ed-789cbee52a87 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:51.802752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbff1b2-9300-4a37-a8e4-acedfe2960ce None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:51.830670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ce97f-7a2f-4227-b9ed-789cbee52a87 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:51.840361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f399f5-d748-43cc-9e8b-260dda98d372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:51.840982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f399f5-d748-43cc-9e8b-260dda98d372 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:51.841265 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f399f5-d748-43cc-9e8b-260dda98d372 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:51.841542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f399f5-d748-43cc-9e8b-260dda98d372 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:51.865118 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261e51b-7eba-4e99-bf09-64bb45a834ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:51.865738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261e51b-7eba-4e99-bf09-64bb45a834ac None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:51.866013 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261e51b-7eba-4e99-bf09-64bb45a834ac None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:51.866282 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261e51b-7eba-4e99-bf09-64bb45a834ac None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:51.866949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261e51b-7eba-4e99-bf09-64bb45a834ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:51.867251 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261e51b-7eba-4e99-bf09-64bb45a834ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:51.868657 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261e51b-7eba-4e99-bf09-64bb45a834ac None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:51.881354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 368/729] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:51.881354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261e51b-7eba-4e99-bf09-64bb45a834ac None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:51.891434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:51.895637 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:51.919558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f399f5-d748-43cc-9e8b-260dda98d372 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:51.921186 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f399f5-d748-43cc-9e8b-260dda98d372 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:51.983779 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261e51b-7eba-4e99-bf09-64bb45a834ac None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:52.022217 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198edbe-d9dc-4fdf-b935-a3072f3dec44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:52.022874 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198edbe-d9dc-4fdf-b935-a3072f3dec44 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:52.023151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198edbe-d9dc-4fdf-b935-a3072f3dec44 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:52.023421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4198edbe-d9dc-4fdf-b935-a3072f3dec44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:52.030054 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 362/730] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 19:13:52.031109 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198edbe-d9dc-4fdf-b935-a3072f3dec44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:52.031727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a939c81-a42b-461e-8983-b31f5ef9333c None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:52.032203 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a939c81-a42b-461e-8983-b31f5ef9333c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:52.032620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4198edbe-d9dc-4fdf-b935-a3072f3dec44 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:52.032835 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a939c81-a42b-461e-8983-b31f5ef9333c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:52.044677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-4198edbe-d9dc-4fdf-b935-a3072f3dec44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:52.044677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:52.044677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:52.044677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:52.044677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:52.044677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:52.044677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:52.044677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:52.044677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:52.044677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:52.044677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:52.044677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:52.044677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:52.044677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:52.044677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:52.045729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:52.045729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:52.045729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:52.045729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:52.045729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:52.045729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:52.045729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:52.045729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:52.045729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:52.045729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:52.045729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:52.045729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:52.045729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:52.045729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:52.045729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:52.045729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:52.045729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:52.046693 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:52.046693 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:52.046693 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:52.046693 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:52.046693 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:52.046693 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:52.046693 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:52.046693 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:52.050691 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 369/731] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:52.051406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198edbe-d9dc-4fdf-b935-a3072f3dec44 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:52.052805 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-2a939c81-a42b-461e-8983-b31f5ef9333c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:52.054195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-2a939c81-a42b-461e-8983-b31f5ef9333c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:52.122803 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:52.126806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 363/732] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:52.127667 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a939c81-a42b-461e-8983-b31f5ef9333c None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:52.140825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:52.168948 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4198edbe-d9dc-4fdf-b935-a3072f3dec44 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:52.208352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b9a16-5a87-4e0d-9ac6-cda5a5ae85d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:52.209325 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b9a16-5a87-4e0d-9ac6-cda5a5ae85d1 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:52.209608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b9a16-5a87-4e0d-9ac6-cda5a5ae85d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:52.209878 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b9a16-5a87-4e0d-9ac6-cda5a5ae85d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:52.221496 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b9a16-5a87-4e0d-9ac6-cda5a5ae85d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:52.222980 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b9a16-5a87-4e0d-9ac6-cda5a5ae85d1 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:52.238545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a939c81-a42b-461e-8983-b31f5ef9333c None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:52.238799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-fd3b9a16-5a87-4e0d-9ac6-cda5a5ae85d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:52.238799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:52.238799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:52.238799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:52.238799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:52.238799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:52.238799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:52.238799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:52.238799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:52.238799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:52.238799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:52.238799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:52.238799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:52.238799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:52.238799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:52.238799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:52.239892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:52.239892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:52.239892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:52.239892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:52.239892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:52.239892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:52.239892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:52.239892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:52.239892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:52.239892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:52.239892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:52.239892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:52.239892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:52.239892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:52.239892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:52.239892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:52.239892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:52.241043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:52.241043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:52.241043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:52.241043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 370/733] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:52.241280 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b9a16-5a87-4e0d-9ac6-cda5a5ae85d1 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:52.280959 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e5246-6481-431f-8be4-120d575de659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:52.281891 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e5246-6481-431f-8be4-120d575de659 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:52.282317 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e5246-6481-431f-8be4-120d575de659 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:52.282747 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e5246-6481-431f-8be4-120d575de659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:52.294297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e5246-6481-431f-8be4-120d575de659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:52.295775 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e5246-6481-431f-8be4-120d575de659 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:52.311267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-1c5e5246-6481-431f-8be4-120d575de659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:52.311267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:52.311267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:52.311267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:52.311267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:52.311267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:52.311267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:52.311267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:52.311267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:52.311267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:52.311267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:52.311267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:52.311267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:52.311267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:52.311267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:52.311267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:52.312291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:52.312291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:52.312291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:52.312291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:52.312291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:52.312291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:52.312291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:52.312291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:52.312291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:52.312291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:52.312291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:52.312291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:52.312291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:52.312291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:52.312291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:52.312291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:52.312291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:52.313268 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:52.313268 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:52.313268 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:52.313268 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 364/734] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:52.313268 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e5246-6481-431f-8be4-120d575de659 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:52.354225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b9a16-5a87-4e0d-9ac6-cda5a5ae85d1 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:52.378259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:52.382898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:52.394623 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38e55f-47ab-42b3-b3a7-3c4791e47741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:52.395199 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38e55f-47ab-42b3-b3a7-3c4791e47741 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:52.395467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38e55f-47ab-42b3-b3a7-3c4791e47741 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:52.395739 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d38e55f-47ab-42b3-b3a7-3c4791e47741 None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:52.396469 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38e55f-47ab-42b3-b3a7-3c4791e47741 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:52.396772 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38e55f-47ab-42b3-b3a7-3c4791e47741 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:52.398236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d38e55f-47ab-42b3-b3a7-3c4791e47741 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:52.410780 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 371/735] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:52.411615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38e55f-47ab-42b3-b3a7-3c4791e47741 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:52.439626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e5246-6481-431f-8be4-120d575de659 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:52.478048 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e455457-81c4-441c-bc03-044b7f45ca4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:52.478831 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e455457-81c4-441c-bc03-044b7f45ca4b None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:52.479489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e455457-81c4-441c-bc03-044b7f45ca4b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:52.479781 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e455457-81c4-441c-bc03-044b7f45ca4b None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:52.480820 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e455457-81c4-441c-bc03-044b7f45ca4b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:52.482261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e455457-81c4-441c-bc03-044b7f45ca4b None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:52.503190 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 365/736] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Aug 7 19:13:52 2022] POST /identity/v3/services => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:13:52.504215 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e455457-81c4-441c-bc03-044b7f45ca4b None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:52.537092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d38e55f-47ab-42b3-b3a7-3c4791e47741 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:52.598165 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d40ed-1409-4906-8098-175ffda3d28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:52.598897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d40ed-1409-4906-8098-175ffda3d28b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:52.599106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d40ed-1409-4906-8098-175ffda3d28b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:52.599368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d40ed-1409-4906-8098-175ffda3d28b None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:52.600045 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d40ed-1409-4906-8098-175ffda3d28b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:52.600348 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d40ed-1409-4906-8098-175ffda3d28b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:52.601760 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d40ed-1409-4906-8098-175ffda3d28b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:52.619107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 372/737] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:52.619107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d40ed-1409-4906-8098-175ffda3d28b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:52.630231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e455457-81c4-441c-bc03-044b7f45ca4b None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:52.656380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:52.668911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:52.677075 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b6920-49c1-47dd-abde-aa7b0356ec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:52.677708 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b6920-49c1-47dd-abde-aa7b0356ec32 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:52.677992 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b6920-49c1-47dd-abde-aa7b0356ec32 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:52.678265 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b6920-49c1-47dd-abde-aa7b0356ec32 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:52.679152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b6920-49c1-47dd-abde-aa7b0356ec32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:52.679289 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b6920-49c1-47dd-abde-aa7b0356ec32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:52.680956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b6920-49c1-47dd-abde-aa7b0356ec32 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:52.696121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 366/738] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:52.697724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e8386-def4-4198-bb67-fac1a1266c34 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:52.698047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e8386-def4-4198-bb67-fac1a1266c34 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:52.698337 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e8386-def4-4198-bb67-fac1a1266c34 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:52.699509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 367/739] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:52.703952 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e8386-def4-4198-bb67-fac1a1266c34 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:52.752318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d40ed-1409-4906-8098-175ffda3d28b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:52.787212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d853c-f746-40d8-b5d0-fb4849c4fad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:52.787820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d853c-f746-40d8-b5d0-fb4849c4fad9 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:52.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d853c-f746-40d8-b5d0-fb4849c4fad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:52.788420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d853c-f746-40d8-b5d0-fb4849c4fad9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:52.789153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d853c-f746-40d8-b5d0-fb4849c4fad9 None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=a4946aec299f45bd8402cff3159d04d7) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:52.789456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d853c-f746-40d8-b5d0-fb4849c4fad9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:52.790892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d853c-f746-40d8-b5d0-fb4849c4fad9 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:52.796694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 373/740] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:13:52 2022] GET /identity/v3/role_assignments?user.id=a4946aec299f45bd8402cff3159d04d7&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=4da42f08d107448fb877b42b61bc74ea => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:52.797503 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d853c-f746-40d8-b5d0-fb4849c4fad9 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:52.818639 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e8386-def4-4198-bb67-fac1a1266c34 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:52.859852 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8d78a-bae6-4dcb-9f1b-557bf1f45002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:52.860918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8d78a-bae6-4dcb-9f1b-557bf1f45002 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:52.861352 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8d78a-bae6-4dcb-9f1b-557bf1f45002 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:52.861769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8d78a-bae6-4dcb-9f1b-557bf1f45002 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:52.870916 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8d78a-bae6-4dcb-9f1b-557bf1f45002 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:52.872491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8d78a-bae6-4dcb-9f1b-557bf1f45002 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:52.886408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-bed8d78a-bae6-4dcb-9f1b-557bf1f45002 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 19:13:52.886408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:52.886408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:52.886408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:52.886408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:52.886408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:52.886408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:52.886408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:52.886408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:52.886408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:52.886408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:52.886408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:52.886408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:52.886408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:52.886408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:52.886408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:52.887424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:52.888405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:52.888405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:52.888405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:52.888405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:52.888405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:52.888405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:52.888405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:52.888405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:52.888405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 19:13:52.888405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:52.888405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 368/741] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:13:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:52.889397 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8585b-18a6-42ba-9625-cd78179375bc None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:52.889645 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8585b-18a6-42ba-9625-cd78179375bc None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:52.889926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b8585b-18a6-42ba-9625-cd78179375bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:52.911124 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-58b8585b-18a6-42ba-9625-cd78179375bc None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:52.912354 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-58b8585b-18a6-42ba-9625-cd78179375bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:52.931933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:52.936990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d853c-f746-40d8-b5d0-fb4849c4fad9 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:52.958225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:52.993246 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7ff4d-3ca9-4381-bbc6-6a1c1939bf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:52.994187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7ff4d-3ca9-4381-bbc6-6a1c1939bf45 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:52.994604 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7ff4d-3ca9-4381-bbc6-6a1c1939bf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:52.995011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7ff4d-3ca9-4381-bbc6-6a1c1939bf45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:52.999056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 369/742] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:52 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 19:13:53.000213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b8585b-18a6-42ba-9625-cd78179375bc None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:53.016152 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7ff4d-3ca9-4381-bbc6-6a1c1939bf45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:53.017686 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe7ff4d-3ca9-4381-bbc6-6a1c1939bf45 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:53.033505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-cbe7ff4d-3ca9-4381-bbc6-6a1c1939bf45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:53.033505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:53.033505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:53.033505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:53.033505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:53.033505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:53.033505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:53.033505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:53.033505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:53.033505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:53.033505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:53.033505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:53.033505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:53.033505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:53.033505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:53.034460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:53.034460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:53.034460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:53.034460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:53.034460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:53.034460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:53.034460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:53.034460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:53.034460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:53.034460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:53.034460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:53.034460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:53.034460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:53.034460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:53.034460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:53.034460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:53.034460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:53.035360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:53.035360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:53.035360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:53.035360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:53.035360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:53.035360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:53.035360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:53.035360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:53.035360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 374/743] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:53.036059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7ff4d-3ca9-4381-bbc6-6a1c1939bf45 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:53.117993 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b8585b-18a6-42ba-9625-cd78179375bc None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:53.158968 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c290547-9ba5-466d-8677-b4085b53888d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:53.159832 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c290547-9ba5-466d-8677-b4085b53888d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:53.160311 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c290547-9ba5-466d-8677-b4085b53888d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:53.160722 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c290547-9ba5-466d-8677-b4085b53888d None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:53.161752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c290547-9ba5-466d-8677-b4085b53888d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:53.162206 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c290547-9ba5-466d-8677-b4085b53888d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:53.163776 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c290547-9ba5-466d-8677-b4085b53888d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:53.182925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 370/744] 10.222.0.17 () {56 vars in 1135 bytes} [Sun Aug 7 19:13:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:53.184164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c290547-9ba5-466d-8677-b4085b53888d None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:53.188073 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7ff4d-3ca9-4381-bbc6-6a1c1939bf45 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:53.224678 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715f392-32c2-437d-a314-ca66b39cfe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:53.225618 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715f392-32c2-437d-a314-ca66b39cfe6f None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:53.226098 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715f392-32c2-437d-a314-ca66b39cfe6f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:53.226548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715f392-32c2-437d-a314-ca66b39cfe6f None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:53.226693 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:53.227877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3715f392-32c2-437d-a314-ca66b39cfe6f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:53.229712 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3715f392-32c2-437d-a314-ca66b39cfe6f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:53.256506 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 375/745] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Aug 7 19:13:53 2022] POST /identity/v3/services => generated 240 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:13:53.258223 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715f392-32c2-437d-a314-ca66b39cfe6f None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:53.301533 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:53.335957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c290547-9ba5-466d-8677-b4085b53888d None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:53.430836 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715f392-32c2-437d-a314-ca66b39cfe6f None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:53.442632 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8e3ae-3d25-4ad0-8f71-7d3c85eb5ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:53.444462 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8e3ae-3d25-4ad0-8f71-7d3c85eb5ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:53.445131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8e3ae-3d25-4ad0-8f71-7d3c85eb5ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:53.445784 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8e3ae-3d25-4ad0-8f71-7d3c85eb5ab3 None admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:53.447147 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8e3ae-3d25-4ad0-8f71-7d3c85eb5ab3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:53.447885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8e3ae-3d25-4ad0-8f71-7d3c85eb5ab3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:53.450457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8e3ae-3d25-4ad0-8f71-7d3c85eb5ab3 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:53.465437 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 371/746] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:53.476043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8e3ae-3d25-4ad0-8f71-7d3c85eb5ab3 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:53.506607 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76bb0a-af4d-43d3-b2ed-6d0853ed4995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:53.507241 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76bb0a-af4d-43d3-b2ed-6d0853ed4995 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:53.507523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76bb0a-af4d-43d3-b2ed-6d0853ed4995 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:53.507799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b76bb0a-af4d-43d3-b2ed-6d0853ed4995 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:53.587086 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:53.594180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76bb0a-af4d-43d3-b2ed-6d0853ed4995 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:53.597122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b76bb0a-af4d-43d3-b2ed-6d0853ed4995 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:53.601843 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8e3ae-3d25-4ad0-8f71-7d3c85eb5ab3 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:53.620950 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:53.656435 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db3a01-a9b6-4064-94e2-d4883277e412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:53.657059 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db3a01-a9b6-4064-94e2-d4883277e412 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:53.657335 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db3a01-a9b6-4064-94e2-d4883277e412 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:53.657601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db3a01-a9b6-4064-94e2-d4883277e412 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:53.658339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db3a01-a9b6-4064-94e2-d4883277e412 None admin] RBAC: Adding query filter params (role.id=43aa4a9830054a09baa1f0aecfd125e2, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=61ff29258a214817b8fc15b51fe8b3e6) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:53.658647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db3a01-a9b6-4064-94e2-d4883277e412 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:53.660114 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db3a01-a9b6-4064-94e2-d4883277e412 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:53.666630 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 372/747] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:13:53 2022] GET /identity/v3/role_assignments?user.id=61ff29258a214817b8fc15b51fe8b3e6&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=43aa4a9830054a09baa1f0aecfd125e2 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:53.741707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 376/748] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:13:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:13:53.747034 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db3a01-a9b6-4064-94e2-d4883277e412 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:53.868122 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db3a01-a9b6-4064-94e2-d4883277e412 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:53.911817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf41adf-0722-421b-aa3d-aa198780fa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:53.912839 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf41adf-0722-421b-aa3d-aa198780fa6d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:53.913295 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf41adf-0722-421b-aa3d-aa198780fa6d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:53.913765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf41adf-0722-421b-aa3d-aa198780fa6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:53.927140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf41adf-0722-421b-aa3d-aa198780fa6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:53.928969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf41adf-0722-421b-aa3d-aa198780fa6d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:53.956410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-cdf41adf-0722-421b-aa3d-aa198780fa6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:53.956410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:53.956410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:53.956410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:53.956410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:53.956410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:53.956410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:53.956410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:53.956410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:53.956410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:53.956410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:53.956410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:53.956410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:53.956410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:53.956410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:53.956410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:53.957422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:53.957422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:53.957422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:53.957422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:53.957422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:53.957422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:53.957422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:53.957422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:53.957422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:53.957422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:53.957422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:53.957422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:53.957422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:53.957422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:53.957422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:53.957422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:53.957422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:53.958315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:53.958315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:53.958315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:53.959897 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 373/749] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:53.965534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76bb0a-af4d-43d3-b2ed-6d0853ed4995 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:54.097470 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b76bb0a-af4d-43d3-b2ed-6d0853ed4995 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:54.124654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:54.139171 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d507484-f501-4c4d-8417-c81bde1dcb52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:54.140244 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d507484-f501-4c4d-8417-c81bde1dcb52 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:54.140984 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d507484-f501-4c4d-8417-c81bde1dcb52 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:54.141260 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d507484-f501-4c4d-8417-c81bde1dcb52 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:54.141924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d507484-f501-4c4d-8417-c81bde1dcb52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:54.142227 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d507484-f501-4c4d-8417-c81bde1dcb52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:54.143714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d507484-f501-4c4d-8417-c81bde1dcb52 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:54.159048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 377/750] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:54.164657 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf41adf-0722-421b-aa3d-aa198780fa6d None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:54.168715 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:54.298470 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf41adf-0722-421b-aa3d-aa198780fa6d None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:54.351100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb411b-e9d3-416a-80a0-416554bf25e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:54.351757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb411b-e9d3-416a-80a0-416554bf25e6 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:54.352055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb411b-e9d3-416a-80a0-416554bf25e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:54.352331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb411b-e9d3-416a-80a0-416554bf25e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:54.359943 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb411b-e9d3-416a-80a0-416554bf25e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:54.361443 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb411b-e9d3-416a-80a0-416554bf25e6 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:54.385466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-a6bb411b-e9d3-416a-80a0-416554bf25e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:54.385466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:54.385466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:54.385466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:54.385466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:54.385466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:54.385466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:54.385466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:54.385466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:54.385466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:54.385466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:54.385466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:54.385466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:54.385466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:54.385466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:54.386472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:54.386472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:54.386472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:54.386472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:54.386472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:54.386472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:54.386472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:54.386472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:54.386472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:54.386472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:54.386472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:54.386472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:54.386472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:54.386472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:54.386472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:54.386472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:54.386472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:54.387973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:54.387973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:54.387973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:54.387973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:54.387973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:54.387973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:54.387973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:54.387973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:54.387973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 374/751] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:54.392496 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d507484-f501-4c4d-8417-c81bde1dcb52 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:54.522649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d507484-f501-4c4d-8417-c81bde1dcb52 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:54.577053 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad7d9e-8345-4870-b8ad-d56050d4af6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:54.577596 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad7d9e-8345-4870-b8ad-d56050d4af6d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:54.577869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad7d9e-8345-4870-b8ad-d56050d4af6d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:54.578133 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad7d9e-8345-4870-b8ad-d56050d4af6d None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:54.578833 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad7d9e-8345-4870-b8ad-d56050d4af6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:54.579130 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad7d9e-8345-4870-b8ad-d56050d4af6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:54.580601 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ad7d9e-8345-4870-b8ad-d56050d4af6d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:54.605617 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 378/752] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:54.611473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb411b-e9d3-416a-80a0-416554bf25e6 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:54.705385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:54.715892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b6e87-e808-4979-8135-54ece0e74db2 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:54.716207 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b6e87-e808-4979-8135-54ece0e74db2 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:54.716504 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b6e87-e808-4979-8135-54ece0e74db2 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:54.718202 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 379/753] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:54.723845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9b998-97b2-48a7-b2dd-bee1861b2bf4 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:54.724158 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9b998-97b2-48a7-b2dd-bee1861b2bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:54.724437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae9b998-97b2-48a7-b2dd-bee1861b2bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:54.758598 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-2ae9b998-97b2-48a7-b2dd-bee1861b2bf4 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:54.759817 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-2ae9b998-97b2-48a7-b2dd-bee1861b2bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:54.779661 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb411b-e9d3-416a-80a0-416554bf25e6 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:54.826351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:54.853765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68faeb-e0f6-4dbf-bfd1-c356772d911d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:54.854283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68faeb-e0f6-4dbf-bfd1-c356772d911d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:54.854563 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68faeb-e0f6-4dbf-bfd1-c356772d911d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:54.854830 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68faeb-e0f6-4dbf-bfd1-c356772d911d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:54.855620 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68faeb-e0f6-4dbf-bfd1-c356772d911d None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=db51971158a946c98a8ed716ecc5d72b) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:54.855925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68faeb-e0f6-4dbf-bfd1-c356772d911d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:54.857716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68faeb-e0f6-4dbf-bfd1-c356772d911d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:54.869088 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 375/754] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:13:54 2022] GET /identity/v3/role_assignments?user.id=db51971158a946c98a8ed716ecc5d72b&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=4da42f08d107448fb877b42b61bc74ea => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:54.935985 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 380/755] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 19:13:54.957383 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68faeb-e0f6-4dbf-bfd1-c356772d911d None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:55.050252 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d19791-d8fb-4287-bc3c-f11ac7349287 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:55.050817 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d19791-d8fb-4287-bc3c-f11ac7349287 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:55.050817 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d19791-d8fb-4287-bc3c-f11ac7349287 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:55.052300 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 381/756] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:55.065867 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03a39e-104b-4ced-877a-dab7b9ae1b25 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:55.066189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03a39e-104b-4ced-877a-dab7b9ae1b25 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:55.066426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03a39e-104b-4ced-877a-dab7b9ae1b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:55.092644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68faeb-e0f6-4dbf-bfd1-c356772d911d None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:55.108097 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-7a03a39e-104b-4ced-877a-dab7b9ae1b25 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:55.109630 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-7a03a39e-104b-4ced-877a-dab7b9ae1b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:55.144610 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:55.158161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a064c-a66d-4765-955a-63b50abaed22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:55.160367 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a064c-a66d-4765-955a-63b50abaed22 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:55.161621 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a064c-a66d-4765-955a-63b50abaed22 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:55.162717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a064c-a66d-4765-955a-63b50abaed22 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:55.163709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a064c-a66d-4765-955a-63b50abaed22 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:55.166282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a064c-a66d-4765-955a-63b50abaed22 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:55.190417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-e00a064c-a66d-4765-955a-63b50abaed22 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 19:13:55.190417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:55.190417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:13:55.190417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:13:55.190417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:13:55.190417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:13:55.190417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:13:55.190417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:13:55.190417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 19:13:55.190417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:55.190417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:13:55.190417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:55.190417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:55.190417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:55.190417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:55.190417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:55.190417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:55.191506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:55.191506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:55.191506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:55.191506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:55.191506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:55.191506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:55.191506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:13:55.191506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:13:55.191506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:13:55.191506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:13:55.191506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:55.191506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:55.191506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:55.191506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:55.191506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:55.191506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:55.192385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:55.192385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:55.192385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:55.192385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:55.192385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:55.192385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:55.192385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:55.192385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:55.192385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:55.192385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:55.192385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:13:55.192385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:13:55.192385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 19:13:55.192385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:55.195350 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 376/757] 10.222.0.17 () {56 vars in 1129 bytes} [Sun Aug 7 19:13:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:55.202962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a064c-a66d-4765-955a-63b50abaed22 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:55.254422 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 382/758] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 19:13:55.289769 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03a39e-104b-4ced-877a-dab7b9ae1b25 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:55.342089 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a064c-a66d-4765-955a-63b50abaed22 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:55.388878 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d3a88-c548-4124-9ecd-fe82ab822652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:55.389435 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d3a88-c548-4124-9ecd-fe82ab822652 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:55.389712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d3a88-c548-4124-9ecd-fe82ab822652 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:55.389989 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d3a88-c548-4124-9ecd-fe82ab822652 None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:55.390636 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d3a88-c548-4124-9ecd-fe82ab822652 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:55.390938 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d3a88-c548-4124-9ecd-fe82ab822652 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:55.392368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d3a88-c548-4124-9ecd-fe82ab822652 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:55.402845 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 377/759] 10.222.0.17 () {56 vars in 1124 bytes} [Sun Aug 7 19:13:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:55.408286 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d3a88-c548-4124-9ecd-fe82ab822652 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:55.496298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:55.509808 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a03a39e-104b-4ced-877a-dab7b9ae1b25 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:55.537081 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:55.555189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080a23b-bf53-4505-a423-140f2711db2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:55.556255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a23b-bf53-4505-a423-140f2711db2b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:55.556859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a23b-bf53-4505-a423-140f2711db2b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:55.557704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080a23b-bf53-4505-a423-140f2711db2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:55.569282 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080a23b-bf53-4505-a423-140f2711db2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:55.574930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-f080a23b-bf53-4505-a423-140f2711db2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:13:55.574930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:55.574930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:55.574930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:55.574930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:55.574930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:55.574930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:55.574930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:55.574930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:55.574930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:55.574930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:55.574930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:55.574930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:55.574930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:55.574930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:55.574930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:55.575861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:55.575861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:55.575861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:55.575861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:55.575861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:55.575861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:55.575861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:55.575861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:55.575861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:55.575861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:55.575861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:55.575861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:55.575861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:55.575861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:55.575861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:55.575861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:55.576722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:55.576722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:55.576722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:55.576722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:55.576722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:55.576722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:55.576722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:55.576722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:55.576722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:55.576722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:55.576722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:13:55.576722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:55.580140 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 383/760] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:13:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:55.585525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080a23b-bf53-4505-a423-140f2711db2b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:55.675085 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d3a88-c548-4124-9ecd-fe82ab822652 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:55.697810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080a23b-bf53-4505-a423-140f2711db2b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:55.737438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600e81f-dea1-4413-b64e-7b98c640d866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:55.738431 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600e81f-dea1-4413-b64e-7b98c640d866 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:55.738961 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600e81f-dea1-4413-b64e-7b98c640d866 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:55.739432 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600e81f-dea1-4413-b64e-7b98c640d866 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:55.740336 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600e81f-dea1-4413-b64e-7b98c640d866 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:55.740850 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600e81f-dea1-4413-b64e-7b98c640d866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:55.742439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600e81f-dea1-4413-b64e-7b98c640d866 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:55.752802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 384/761] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:55.754069 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7943b-eeca-4014-b17c-bf4f3e1b1df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:55.754720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7943b-eeca-4014-b17c-bf4f3e1b1df0 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:55.754997 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7943b-eeca-4014-b17c-bf4f3e1b1df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:55.755269 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7943b-eeca-4014-b17c-bf4f3e1b1df0 None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:55.755913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7943b-eeca-4014-b17c-bf4f3e1b1df0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:55.756247 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7943b-eeca-4014-b17c-bf4f3e1b1df0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:55.757622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7943b-eeca-4014-b17c-bf4f3e1b1df0 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:55.758005 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600e81f-dea1-4413-b64e-7b98c640d866 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:55.767588 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 378/762] 10.222.0.17 () {56 vars in 1124 bytes} [Sun Aug 7 19:13:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:55.777193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7943b-eeca-4014-b17c-bf4f3e1b1df0 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:55.780352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:55.869287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600e81f-dea1-4413-b64e-7b98c640d866 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:55.906959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa089b-85d0-4106-a1a4-dd6f85355f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:55.908079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa089b-85d0-4106-a1a4-dd6f85355f95 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:55.908421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7943b-eeca-4014-b17c-bf4f3e1b1df0 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:55.908839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa089b-85d0-4106-a1a4-dd6f85355f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:55.909323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa089b-85d0-4106-a1a4-dd6f85355f95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:55.921253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aa089b-85d0-4106-a1a4-dd6f85355f95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:55.923263 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aa089b-85d0-4106-a1a4-dd6f85355f95 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:55.929807 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:55.941479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-20aa089b-85d0-4106-a1a4-dd6f85355f95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:55.941479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:55.941479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:55.941479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:55.941479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:55.941479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:55.941479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:55.941479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:55.941479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:55.941479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:55.941479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:55.941479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:55.941479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:55.941479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:55.941479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:55.941479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:55.942523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:55.942523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:55.942523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:55.942523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:55.942523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:55.942523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:55.942523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:55.942523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:55.942523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:55.942523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:55.942523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:55.942523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:55.942523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:55.942523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:55.942523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:55.942523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:55.942523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:55.943419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:55.943419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:55.943419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:55.945943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 385/763] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:13:55.947457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c3e79a-903b-45d6-b644-a15cdd17b79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:55.947931 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff59506-f4c1-4a7d-8895-2390e8d0b0e7 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:55.948171 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3e79a-903b-45d6-b644-a15cdd17b79a None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:55.948394 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3e79a-903b-45d6-b644-a15cdd17b79a None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:55.948660 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c3e79a-903b-45d6-b644-a15cdd17b79a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:55.949113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff59506-f4c1-4a7d-8895-2390e8d0b0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:55.949412 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c3e79a-903b-45d6-b644-a15cdd17b79a None admin] RBAC: Adding query filter params (interface=public, service_id=2d908dd757be40a091c4795d52f789a3, region_id=RegionOne) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:55.949717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c3e79a-903b-45d6-b644-a15cdd17b79a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:55.950099 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff59506-f4c1-4a7d-8895-2390e8d0b0e7 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:55.951147 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c3e79a-903b-45d6-b644-a15cdd17b79a None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:55.952087 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 386/764] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:55.953184 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff59506-f4c1-4a7d-8895-2390e8d0b0e7 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:55.961197 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 379/765] 10.222.0.17 () {56 vars in 1251 bytes} [Sun Aug 7 19:13:55 2022] GET /identity/v3/endpoints?service_id=2d908dd757be40a091c4795d52f789a3&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:55.962457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6de005-5adf-4e09-b800-88017cde3aec None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:55.962737 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6de005-5adf-4e09-b800-88017cde3aec None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:55.963005 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6de005-5adf-4e09-b800-88017cde3aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:55.982678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-1b6de005-5adf-4e09-b800-88017cde3aec None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:55.983981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1b6de005-5adf-4e09-b800-88017cde3aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:56.041845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:56.058373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 380/766] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:13:56.059303 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6de005-5adf-4e09-b800-88017cde3aec None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:56.077520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff59506-f4c1-4a7d-8895-2390e8d0b0e7 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:56.114565 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4713136-89b1-4660-a8f5-4dec47b6c882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:56.115588 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4713136-89b1-4660-a8f5-4dec47b6c882 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:56.116114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4713136-89b1-4660-a8f5-4dec47b6c882 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:56.116586 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4713136-89b1-4660-a8f5-4dec47b6c882 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:56.117464 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4713136-89b1-4660-a8f5-4dec47b6c882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:56.117963 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4713136-89b1-4660-a8f5-4dec47b6c882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:56.119563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4713136-89b1-4660-a8f5-4dec47b6c882 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:56.132531 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 387/767] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:56.133842 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4713136-89b1-4660-a8f5-4dec47b6c882 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:56.169255 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6de005-5adf-4e09-b800-88017cde3aec None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:56.185627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:56.210805 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58154566-7503-4f2c-a363-0e19ec19762e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:56.211430 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58154566-7503-4f2c-a363-0e19ec19762e None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:56.211738 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58154566-7503-4f2c-a363-0e19ec19762e None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:56.212149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58154566-7503-4f2c-a363-0e19ec19762e None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:56.212734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58154566-7503-4f2c-a363-0e19ec19762e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:56.214310 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58154566-7503-4f2c-a363-0e19ec19762e None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:56.220555 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 381/768] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Aug 7 19:13:56 2022] GET /identity/v3/services => generated 772 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:56.221529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58154566-7503-4f2c-a363-0e19ec19762e None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:56.267679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4713136-89b1-4660-a8f5-4dec47b6c882 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:56.319062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:56.325503 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed53924-c20d-4cdf-8beb-096c0427f88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:56.326543 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed53924-c20d-4cdf-8beb-096c0427f88d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:56.327029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed53924-c20d-4cdf-8beb-096c0427f88d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:56.327527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed53924-c20d-4cdf-8beb-096c0427f88d None admin] PATH_INFO: `/v3/services/network` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:56.328368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed53924-c20d-4cdf-8beb-096c0427f88d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:56.330124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed53924-c20d-4cdf-8beb-096c0427f88d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:56.340685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-8ed53924-c20d-4cdf-8beb-096c0427f88d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 19:13:56.340685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:56.340685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:13:56.340685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:13:56.340685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:13:56.340685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:13:56.340685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:13:56.340685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:13:56.340685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 19:13:56.340685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:56.340685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:13:56.340685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:56.340685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:56.340685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:56.340685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:56.340685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:56.340685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:56.341741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:56.341741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:56.341741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:56.341741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:56.341741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:56.341741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:56.341741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:13:56.341741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:13:56.341741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:13:56.341741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:13:56.341741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:56.341741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:56.341741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:56.341741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:56.341741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:56.341741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:56.342557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:56.342557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:56.342557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:56.342557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:56.342557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:56.342557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:56.342557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:56.342557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:56.342557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:56.342557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:56.342557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:13:56.342557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:13:56.342557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 19:13:56.342557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:56.343373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58154566-7503-4f2c-a363-0e19ec19762e None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:56.345741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 388/769] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:56 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:56.355661 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed53924-c20d-4cdf-8beb-096c0427f88d None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:56.387240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faccfd5-f16b-49b2-8c2b-0de0f28c4e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:56.387240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faccfd5-f16b-49b2-8c2b-0de0f28c4e0c None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:56.387240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faccfd5-f16b-49b2-8c2b-0de0f28c4e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:56.387240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faccfd5-f16b-49b2-8c2b-0de0f28c4e0c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:56.397764 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faccfd5-f16b-49b2-8c2b-0de0f28c4e0c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:56.399029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faccfd5-f16b-49b2-8c2b-0de0f28c4e0c None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:56.416944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-6faccfd5-f16b-49b2-8c2b-0de0f28c4e0c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:13:56.416944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:56.416944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:56.416944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:56.416944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:56.416944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:56.416944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:56.416944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:56.416944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:56.416944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:56.416944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:56.416944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:56.416944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:56.416944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:56.416944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:56.416944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:56.418003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:56.419529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:56.419529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:56.419529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:56.419529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:56.419529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:56.419529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:56.419529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:56.419529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:56.419529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:13:56.419529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:56.419529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 382/770] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Aug 7 19:13:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:56.423772 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faccfd5-f16b-49b2-8c2b-0de0f28c4e0c None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:56.491395 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed53924-c20d-4cdf-8beb-096c0427f88d None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:56.512399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:56.552419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de0d41-bea1-4084-b1bf-aff0fb73f452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:56.553378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de0d41-bea1-4084-b1bf-aff0fb73f452 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:56.553658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de0d41-bea1-4084-b1bf-aff0fb73f452 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:56.553931 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de0d41-bea1-4084-b1bf-aff0fb73f452 None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:56.554580 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de0d41-bea1-4084-b1bf-aff0fb73f452 None admin] RBAC: Adding query filter params (name=network) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:56.554883 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de0d41-bea1-4084-b1bf-aff0fb73f452 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:56.559002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de0d41-bea1-4084-b1bf-aff0fb73f452 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:56.569410 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 389/771] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Aug 7 19:13:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:56.576665 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de0d41-bea1-4084-b1bf-aff0fb73f452 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:56.588986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faccfd5-f16b-49b2-8c2b-0de0f28c4e0c None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:56.648660 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8de577-5671-4ba1-a9b8-0b595dfe2e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:56.653428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8de577-5671-4ba1-a9b8-0b595dfe2e95 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:56.653711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8de577-5671-4ba1-a9b8-0b595dfe2e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:56.654834 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8de577-5671-4ba1-a9b8-0b595dfe2e95 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:56.655594 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8de577-5671-4ba1-a9b8-0b595dfe2e95 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:56.655906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8de577-5671-4ba1-a9b8-0b595dfe2e95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:56.657390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8de577-5671-4ba1-a9b8-0b595dfe2e95 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:56.672909 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:56.679473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 383/772] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Aug 7 19:13:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:56.680945 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5687cd-db78-47e7-9054-9fed1e8c593c None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:56.681234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5687cd-db78-47e7-9054-9fed1e8c593c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:56.681510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5687cd-db78-47e7-9054-9fed1e8c593c None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:56.687424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 384/773] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:56.687424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5687cd-db78-47e7-9054-9fed1e8c593c None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:56.735180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de0d41-bea1-4084-b1bf-aff0fb73f452 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:56.778311 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec400c5b-51aa-4b1e-889a-833e0b9a35ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:56.779205 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec400c5b-51aa-4b1e-889a-833e0b9a35ac None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:56.779315 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec400c5b-51aa-4b1e-889a-833e0b9a35ac None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:56.779524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec400c5b-51aa-4b1e-889a-833e0b9a35ac None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:56.780233 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec400c5b-51aa-4b1e-889a-833e0b9a35ac None admin] RBAC: Adding query filter params (type=network) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:56.780537 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec400c5b-51aa-4b1e-889a-833e0b9a35ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:56.782057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec400c5b-51aa-4b1e-889a-833e0b9a35ac None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:56.794337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 390/774] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Aug 7 19:13:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:56.794337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91448d16-b09f-44ee-b1c2-a84691cf2850 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:56.794337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91448d16-b09f-44ee-b1c2-a84691cf2850 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:56.794337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91448d16-b09f-44ee-b1c2-a84691cf2850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:56.804379 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:56.811021 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5687cd-db78-47e7-9054-9fed1e8c593c None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:56.811299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-91448d16-b09f-44ee-b1c2-a84691cf2850 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:56.812542 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-91448d16-b09f-44ee-b1c2-a84691cf2850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:56.859968 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c9054-1dd4-4f44-b002-1cc3c9c88aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:56.860649 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c9054-1dd4-4f44-b002-1cc3c9c88aec None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:56.860945 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c9054-1dd4-4f44-b002-1cc3c9c88aec None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:56.861203 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c9054-1dd4-4f44-b002-1cc3c9c88aec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:56.872081 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c9054-1dd4-4f44-b002-1cc3c9c88aec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:56.873628 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c9054-1dd4-4f44-b002-1cc3c9c88aec None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:56.889906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-517c9054-1dd4-4f44-b002-1cc3c9c88aec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:56.889906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:56.889906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:56.889906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:56.889906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:56.889906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:56.889906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:56.889906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:56.889906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:56.889906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:56.889906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:56.889906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:56.889906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:56.889906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:56.889906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:56.889906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:56.890913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:56.890913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:56.890913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:56.890913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:56.890913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:56.890913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:56.890913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:56.890913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:56.890913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:56.890913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:56.890913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:56.890913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:56.890913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:56.890913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:56.890913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:56.890913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:56.890913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:56.891813 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:56.891813 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:56.891813 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:56.891813 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 385/775] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:56.892516 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c9054-1dd4-4f44-b002-1cc3c9c88aec None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:56.908572 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 391/776] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:13:56.908572 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91448d16-b09f-44ee-b1c2-a84691cf2850 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:56.995213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:57.030458 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c9054-1dd4-4f44-b002-1cc3c9c88aec None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:57.073116 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e892c9d-7416-4e26-a48b-6c25b9467831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:57.074101 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e892c9d-7416-4e26-a48b-6c25b9467831 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:57.074598 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e892c9d-7416-4e26-a48b-6c25b9467831 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:57.074978 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e892c9d-7416-4e26-a48b-6c25b9467831 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:57.075774 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e892c9d-7416-4e26-a48b-6c25b9467831 None admin] RBAC: Adding query filter params (interface=public, service_id=37d7193485b24ec0ac4a90e70631f6a5, region_id=RegionOne) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:57.076201 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e892c9d-7416-4e26-a48b-6c25b9467831 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:57.077913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e892c9d-7416-4e26-a48b-6c25b9467831 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:57.089559 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 386/777] 10.222.0.17 () {56 vars in 1251 bytes} [Sun Aug 7 19:13:56 2022] GET /identity/v3/endpoints?service_id=37d7193485b24ec0ac4a90e70631f6a5&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:57.090709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e892c9d-7416-4e26-a48b-6c25b9467831 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:57.095954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91448d16-b09f-44ee-b1c2-a84691cf2850 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:57.109805 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:57.135618 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6f56c-ad7f-40d8-be75-97ac27f76d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:57.136663 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6f56c-ad7f-40d8-be75-97ac27f76d9a None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:57.137163 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6f56c-ad7f-40d8-be75-97ac27f76d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:57.137679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6f56c-ad7f-40d8-be75-97ac27f76d9a None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:57.138525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6f56c-ad7f-40d8-be75-97ac27f76d9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:57.139051 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6f56c-ad7f-40d8-be75-97ac27f76d9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:57.140708 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6f56c-ad7f-40d8-be75-97ac27f76d9a None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:57.154029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 392/778] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:57.155264 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6f56c-ad7f-40d8-be75-97ac27f76d9a None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:57.212930 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e892c9d-7416-4e26-a48b-6c25b9467831 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:57.251638 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910d078-da2c-402f-a294-15d762a5d844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:57.252302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910d078-da2c-402f-a294-15d762a5d844 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:57.252577 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910d078-da2c-402f-a294-15d762a5d844 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:57.252848 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910d078-da2c-402f-a294-15d762a5d844 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:57.263141 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910d078-da2c-402f-a294-15d762a5d844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:57.264123 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6f56c-ad7f-40d8-be75-97ac27f76d9a None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:57.268208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-0910d078-da2c-402f-a294-15d762a5d844 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:57.268208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:57.268208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:57.268208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:57.268208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:57.268208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:57.268208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:57.268208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:57.268208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:57.268208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:57.268208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:57.268208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:57.268208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:57.268208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:57.268208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:57.268208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:57.269180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:57.269180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:57.269180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:57.269180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:57.269180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:57.269180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:57.269180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:57.269180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:57.269180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:57.269180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:57.269180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:57.269180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:57.269180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:57.269180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:57.269180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:57.269180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:57.270029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:57.270029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:57.270029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:57.270029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:57.270029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:57.270029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:57.270029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:57.270029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:57.270029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:57.270029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:57.270029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:13:57.270029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:57.270029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 387/779] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:13:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:57.270737 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910d078-da2c-402f-a294-15d762a5d844 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:57.283403 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:57.313744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b12e6e-7e00-482c-9ae9-2078c08d9d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:57.314424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b12e6e-7e00-482c-9ae9-2078c08d9d61 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:57.314703 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b12e6e-7e00-482c-9ae9-2078c08d9d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:57.314971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b12e6e-7e00-482c-9ae9-2078c08d9d61 None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:57.315941 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b12e6e-7e00-482c-9ae9-2078c08d9d61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:57.317992 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b12e6e-7e00-482c-9ae9-2078c08d9d61 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:57.331895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 393/780] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Aug 7 19:13:57 2022] GET /identity/v3/services => generated 772 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:57.331895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d2861-b041-476d-9d79-396e90448c6a None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:57.331895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d2861-b041-476d-9d79-396e90448c6a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:57.331895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d2861-b041-476d-9d79-396e90448c6a None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:57.331895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 394/781] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:57.331895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d2861-b041-476d-9d79-396e90448c6a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:57.359069 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:57.388456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910d078-da2c-402f-a294-15d762a5d844 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:57.434059 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35995418-caa5-4cae-a346-c7f58ea954ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:57.435220 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35995418-caa5-4cae-a346-c7f58ea954ca None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:57.435506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35995418-caa5-4cae-a346-c7f58ea954ca None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:57.435917 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35995418-caa5-4cae-a346-c7f58ea954ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:57.445889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35995418-caa5-4cae-a346-c7f58ea954ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:57.447852 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35995418-caa5-4cae-a346-c7f58ea954ca None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:57.455959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d2861-b041-476d-9d79-396e90448c6a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:57.463732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-35995418-caa5-4cae-a346-c7f58ea954ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:57.463732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:57.463732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:57.463732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:57.463732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:57.463732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:57.463732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:57.463732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:57.463732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:57.463732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:57.463732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:57.463732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:57.463732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:57.463732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:57.463732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:57.464785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:57.464785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:57.464785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:57.464785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:57.464785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:57.464785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:57.464785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:57.464785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:57.464785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:57.464785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:57.464785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:57.464785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:57.464785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:57.464785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:57.464785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:57.464785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:57.464785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:57.465668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:57.465668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:57.465668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:57.465668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:57.465668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:57.465668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:57.465668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:57.465668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:57.466612 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 388/782] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:57.467777 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92b401-92f1-48bf-afcd-c070f16e6bcd None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:57.468459 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92b401-92f1-48bf-afcd-c070f16e6bcd None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:57.468738 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92b401-92f1-48bf-afcd-c070f16e6bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:57.497407 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c687a-21ee-4ac2-9a9e-54c7a7c561b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:57.497779 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-ee92b401-92f1-48bf-afcd-c070f16e6bcd None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:57.498033 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c687a-21ee-4ac2-9a9e-54c7a7c561b8 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:57.498308 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c687a-21ee-4ac2-9a9e-54c7a7c561b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:57.498574 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c687a-21ee-4ac2-9a9e-54c7a7c561b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:57.499007 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ee92b401-92f1-48bf-afcd-c070f16e6bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:57.499252 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c687a-21ee-4ac2-9a9e-54c7a7c561b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:57.499553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c687a-21ee-4ac2-9a9e-54c7a7c561b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:57.501073 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c687a-21ee-4ac2-9a9e-54c7a7c561b8 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:57.511379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 395/783] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:13:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:57.512302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c687a-21ee-4ac2-9a9e-54c7a7c561b8 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:57.561296 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:57.591052 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 389/784] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:13:57.596477 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92b401-92f1-48bf-afcd-c070f16e6bcd None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:57.642395 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c687a-21ee-4ac2-9a9e-54c7a7c561b8 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:57.672189 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:57.680215 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97102f4-8e1a-4a0d-b2ef-ce1cfac49068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:57.680852 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97102f4-8e1a-4a0d-b2ef-ce1cfac49068 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:57.681130 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97102f4-8e1a-4a0d-b2ef-ce1cfac49068 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:57.681403 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97102f4-8e1a-4a0d-b2ef-ce1cfac49068 None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:57.682452 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97102f4-8e1a-4a0d-b2ef-ce1cfac49068 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:57.682762 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97102f4-8e1a-4a0d-b2ef-ce1cfac49068 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:57.684924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97102f4-8e1a-4a0d-b2ef-ce1cfac49068 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:57.698749 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 396/785] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:57.699786 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97102f4-8e1a-4a0d-b2ef-ce1cfac49068 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:57.718143 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92b401-92f1-48bf-afcd-c070f16e6bcd None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:57.756686 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254b096-2797-4ce7-94d0-984e8876a709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:57.757591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254b096-2797-4ce7-94d0-984e8876a709 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:57.758036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254b096-2797-4ce7-94d0-984e8876a709 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:57.758448 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9254b096-2797-4ce7-94d0-984e8876a709 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:57.769487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254b096-2797-4ce7-94d0-984e8876a709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:57.771164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9254b096-2797-4ce7-94d0-984e8876a709 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:57.786405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-9254b096-2797-4ce7-94d0-984e8876a709 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:57.786405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:57.786405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:57.786405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:57.786405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:57.786405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:57.786405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:57.786405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:57.786405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:57.786405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:57.786405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:57.786405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:57.786405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:57.786405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:57.786405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:57.786405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:57.787580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:57.787580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:57.787580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:57.787580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:57.787580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:57.787580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:57.787580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:57.787580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:57.787580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:57.787580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:57.787580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:57.787580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:57.787580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:57.787580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:57.787580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:57.787580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:57.787580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:57.788652 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:57.788652 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:57.788652 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:57.794752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 390/786] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:57.795850 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254b096-2797-4ce7-94d0-984e8876a709 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:57.804509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:57.809322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97102f4-8e1a-4a0d-b2ef-ce1cfac49068 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:57.845798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96950710-e5cd-4bbd-bf4e-68bcc5fa6d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:57.846354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96950710-e5cd-4bbd-bf4e-68bcc5fa6d3a None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:57.846679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96950710-e5cd-4bbd-bf4e-68bcc5fa6d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:57.846897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96950710-e5cd-4bbd-bf4e-68bcc5fa6d3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:57.858111 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96950710-e5cd-4bbd-bf4e-68bcc5fa6d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:13:57.863400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-96950710-e5cd-4bbd-bf4e-68bcc5fa6d3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:13:57.863400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:57.863400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:57.863400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:57.863400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:57.863400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:57.863400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:57.863400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:57.863400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:57.863400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:57.863400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:57.863400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:57.863400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:13:57.863400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:13:57.863400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:13:57.863400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:13:57.864378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:13:57.864378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:13:57.864378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:13:57.864378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:13:57.864378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:57.864378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:57.864378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:57.864378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:57.864378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:57.864378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:57.864378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:57.864378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:57.864378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:57.864378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:57.864378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:57.864378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:57.865929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:57.865929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:57.865929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:57.865929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:57.865929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:13:57.865929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:13:57.865929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:13:57.865929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:13:57.865929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:13:57.865929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:13:57.865929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:13:57.865929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:57.865929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 397/787] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:13:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:57.865929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96950710-e5cd-4bbd-bf4e-68bcc5fa6d3a None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:57.910915 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9254b096-2797-4ce7-94d0-984e8876a709 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:57.926063 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:57.948320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cd966-7a95-458f-a10a-58af58bbc2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:57.949429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cd966-7a95-458f-a10a-58af58bbc2bc None admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:57.949903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cd966-7a95-458f-a10a-58af58bbc2bc None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:57.950361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cd966-7a95-458f-a10a-58af58bbc2bc None admin] PATH_INFO: `/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/a4946aec299f45bd8402cff3159d04d7/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:57.953621 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cd966-7a95-458f-a10a-58af58bbc2bc None admin] RBAC: Authorizing `identity:create_grant(project_id=37703d50cd7e4d9bbdc797a7a1466b66, user_id=a4946aec299f45bd8402cff3159d04d7, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:57.955779 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023cd966-7a95-458f-a10a-58af58bbc2bc None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:57.974003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 391/788] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:13:57 2022] PUT /identity/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/a4946aec299f45bd8402cff3159d04d7/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:13:57.975454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cd966-7a95-458f-a10a-58af58bbc2bc None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:57.980922 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96950710-e5cd-4bbd-bf4e-68bcc5fa6d3a None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:58.020381 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f5a73-f6c9-4b80-aa1a-08c9b26683cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:58.021231 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f5a73-f6c9-4b80-aa1a-08c9b26683cd None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:58.021629 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f5a73-f6c9-4b80-aa1a-08c9b26683cd None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:58.021996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f5a73-f6c9-4b80-aa1a-08c9b26683cd None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:58.022751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f5a73-f6c9-4b80-aa1a-08c9b26683cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:58.023201 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f5a73-f6c9-4b80-aa1a-08c9b26683cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:58.025103 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f5a73-f6c9-4b80-aa1a-08c9b26683cd None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:58.039710 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 398/789] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:58.049373 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f5a73-f6c9-4b80-aa1a-08c9b26683cd None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:58.083679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:58.107294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cd966-7a95-458f-a10a-58af58bbc2bc None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:58.144261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5d8f7-7833-4a47-aece-af7b77d22f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:58.145297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5d8f7-7833-4a47-aece-af7b77d22f35 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:58.145786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5d8f7-7833-4a47-aece-af7b77d22f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:58.146249 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae5d8f7-7833-4a47-aece-af7b77d22f35 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:58.147204 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5d8f7-7833-4a47-aece-af7b77d22f35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:58.147720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5d8f7-7833-4a47-aece-af7b77d22f35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:58.149366 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae5d8f7-7833-4a47-aece-af7b77d22f35 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:58.159546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 392/790] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:13:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:58.165245 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5d8f7-7833-4a47-aece-af7b77d22f35 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:58.208180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:58.249812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f5a73-f6c9-4b80-aa1a-08c9b26683cd None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:58.289368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207e37d-17cb-4759-811e-29b86d186e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:58.290099 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207e37d-17cb-4759-811e-29b86d186e2c None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:58.290459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207e37d-17cb-4759-811e-29b86d186e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:58.290838 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207e37d-17cb-4759-811e-29b86d186e2c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:58.293246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae5d8f7-7833-4a47-aece-af7b77d22f35 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:58.299541 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207e37d-17cb-4759-811e-29b86d186e2c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:58.301129 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207e37d-17cb-4759-811e-29b86d186e2c None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:58.316314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-7207e37d-17cb-4759-811e-29b86d186e2c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 19:13:58.316314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:58.316314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:58.316314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:58.316314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:58.316314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:58.316314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:58.316314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:58.316314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:58.316314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:58.316314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:58.316314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:58.316314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:58.316314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:58.316314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:58.316314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:58.317314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:58.318602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:58.318602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:58.318602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:58.318602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:58.318602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:58.318602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:58.318602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:58.318602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:58.318602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 19:13:58.318602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:58.318602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 399/791] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Aug 7 19:13:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:58.329323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207e37d-17cb-4759-811e-29b86d186e2c None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:58.338150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962b09d-54db-4f11-97f9-622d6f1fffa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:58.339014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962b09d-54db-4f11-97f9-622d6f1fffa5 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:58.339486 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962b09d-54db-4f11-97f9-622d6f1fffa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:58.339915 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4962b09d-54db-4f11-97f9-622d6f1fffa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:58.351308 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4962b09d-54db-4f11-97f9-622d6f1fffa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:58.353150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4962b09d-54db-4f11-97f9-622d6f1fffa5 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:58.383188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-4962b09d-54db-4f11-97f9-622d6f1fffa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:58.383188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:58.383188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:58.383188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:58.383188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:58.383188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:58.383188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:58.383188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:58.383188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:58.383188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:58.383188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:58.383188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:58.383188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:58.383188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:58.383188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:58.383188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:58.384185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:58.384185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:58.384185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:58.384185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:58.384185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:58.384185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:58.384185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:58.384185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:58.384185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:58.384185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:58.384185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:58.384185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:58.384185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:58.384185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:58.384185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:58.384185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:58.384185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:58.385078 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:58.385078 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:58.385078 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:58.393961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 393/792] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:58.399579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962b09d-54db-4f11-97f9-622d6f1fffa5 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:58.463350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:58.486564 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207e37d-17cb-4759-811e-29b86d186e2c None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:58.522075 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38565ff-6b4d-48eb-88d5-3aea42152ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:58.523088 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38565ff-6b4d-48eb-88d5-3aea42152ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:58.523605 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38565ff-6b4d-48eb-88d5-3aea42152ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:58.524226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38565ff-6b4d-48eb-88d5-3aea42152ea7 None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:58.525175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38565ff-6b4d-48eb-88d5-3aea42152ea7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:58.525680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38565ff-6b4d-48eb-88d5-3aea42152ea7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:58.527371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38565ff-6b4d-48eb-88d5-3aea42152ea7 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:58.538708 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:58.547271 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 400/793] 10.222.0.17 () {56 vars in 1149 bytes} [Sun Aug 7 19:13:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:58.548979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d14d7-2f9d-4ada-b077-b2c5adc1757f None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:58.549457 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d14d7-2f9d-4ada-b077-b2c5adc1757f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:58.549955 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d14d7-2f9d-4ada-b077-b2c5adc1757f None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:58.551679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 401/794] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:58.552588 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d14d7-2f9d-4ada-b077-b2c5adc1757f None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:58.556796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4962b09d-54db-4f11-97f9-622d6f1fffa5 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:58.606527 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011970e2-2501-4fb9-88eb-85f19ba02f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:58.607154 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011970e2-2501-4fb9-88eb-85f19ba02f8b None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:58.607570 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011970e2-2501-4fb9-88eb-85f19ba02f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:58.607985 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011970e2-2501-4fb9-88eb-85f19ba02f8b None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:58.608656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011970e2-2501-4fb9-88eb-85f19ba02f8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:58.608958 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011970e2-2501-4fb9-88eb-85f19ba02f8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:58.610974 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011970e2-2501-4fb9-88eb-85f19ba02f8b None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:58.628675 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 394/795] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:58.630330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff92b32-b6a2-4bde-a58c-ec6ed66e46e0 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:58.631225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff92b32-b6a2-4bde-a58c-ec6ed66e46e0 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:58.631500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff92b32-b6a2-4bde-a58c-ec6ed66e46e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:58.660081 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-5ff92b32-b6a2-4bde-a58c-ec6ed66e46e0 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:58.661486 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5ff92b32-b6a2-4bde-a58c-ec6ed66e46e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:58.669429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d14d7-2f9d-4ada-b077-b2c5adc1757f None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:58.710142 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064d11e-92ab-44a2-9291-ca220d8b63ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:58.710773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064d11e-92ab-44a2-9291-ca220d8b63ea None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:58.711050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064d11e-92ab-44a2-9291-ca220d8b63ea None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:58.711315 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064d11e-92ab-44a2-9291-ca220d8b63ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:58.714767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:58.726254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064d11e-92ab-44a2-9291-ca220d8b63ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:58.728301 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064d11e-92ab-44a2-9291-ca220d8b63ea None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:58.746944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-b064d11e-92ab-44a2-9291-ca220d8b63ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:58.746944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:58.746944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:58.746944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:58.746944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:58.746944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:58.746944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:58.746944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:58.746944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:58.746944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:58.746944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:58.746944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:58.746944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:58.746944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:58.746944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:58.746944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:58.747899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:58.747899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:58.747899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:58.747899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:58.747899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:58.747899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:58.747899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:58.747899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:58.747899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:58.747899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:58.747899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:58.747899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:58.747899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:58.747899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:58.747899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:58.747899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:58.747899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:58.748823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:58.748823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:58.748823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:58.750580 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 402/796] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:58.750979 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 395/797] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:13:58.752323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064d11e-92ab-44a2-9291-ca220d8b63ea None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:58.756402 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff92b32-b6a2-4bde-a58c-ec6ed66e46e0 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:58.846532 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:58.871363 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff92b32-b6a2-4bde-a58c-ec6ed66e46e0 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:58.872602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064d11e-92ab-44a2-9291-ca220d8b63ea None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:58.907277 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64164109-11e2-4a31-a156-07dd1b5049a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:58.907574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f7cb2-d4de-4142-8745-5572b0078180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:58.908329 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f7cb2-d4de-4142-8745-5572b0078180 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:58.908527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64164109-11e2-4a31-a156-07dd1b5049a0 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:58.908783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f7cb2-d4de-4142-8745-5572b0078180 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:58.909124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64164109-11e2-4a31-a156-07dd1b5049a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:58.909241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f7cb2-d4de-4142-8745-5572b0078180 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:58.909628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64164109-11e2-4a31-a156-07dd1b5049a0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:58.910030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f7cb2-d4de-4142-8745-5572b0078180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:58.910468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f7cb2-d4de-4142-8745-5572b0078180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:58.911981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f7cb2-d4de-4142-8745-5572b0078180 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:58.918299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64164109-11e2-4a31-a156-07dd1b5049a0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:58.919914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64164109-11e2-4a31-a156-07dd1b5049a0 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:58.924682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 396/798] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:13:58.926247 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f7cb2-d4de-4142-8745-5572b0078180 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:58.932861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-64164109-11e2-4a31-a156-07dd1b5049a0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 19:13:58.932861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:58.932861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:58.932861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:58.932861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:58.932861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:58.932861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:58.932861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:58.932861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:58.932861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:58.932861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:58.932861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:58.932861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:13:58.932861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:13:58.932861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:13:58.932861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:58.933877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:58.934805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:58.934805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:58.934805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:13:58.934805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:13:58.934805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:13:58.934805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:13:58.934805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:13:58.934805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:13:58.934805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 19:13:58.934805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:58.936985 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 403/799] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:13:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:58.937998 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64164109-11e2-4a31-a156-07dd1b5049a0 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:58.988258 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:59.043412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64164109-11e2-4a31-a156-07dd1b5049a0 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:59.047369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f7cb2-d4de-4142-8745-5572b0078180 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:59.083927 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3a604-4d18-4f9f-8841-6a9e394aeb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:59.084815 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3a604-4d18-4f9f-8841-6a9e394aeb1b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:59.085158 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3a604-4d18-4f9f-8841-6a9e394aeb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:59.085493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3a604-4d18-4f9f-8841-6a9e394aeb1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:59.085775 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667d013-d53f-4202-8c6d-0176c5d9a4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:59.086590 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667d013-d53f-4202-8c6d-0176c5d9a4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:59.087032 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667d013-d53f-4202-8c6d-0176c5d9a4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:59.087465 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667d013-d53f-4202-8c6d-0176c5d9a4a6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:59.088293 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667d013-d53f-4202-8c6d-0176c5d9a4a6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:59.089905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667d013-d53f-4202-8c6d-0176c5d9a4a6 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:59.094045 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d3a604-4d18-4f9f-8841-6a9e394aeb1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:59.095510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d3a604-4d18-4f9f-8841-6a9e394aeb1b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:59.099431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-d667d013-d53f-4202-8c6d-0176c5d9a4a6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 19:13:59.099431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:59.099431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:13:59.099431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:13:59.099431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:13:59.099431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:13:59.099431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:13:59.099431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:13:59.099431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 19:13:59.099431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:59.099431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:13:59.099431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:59.099431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:59.099431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:59.099431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:59.099431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:59.099431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:59.100445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:59.100445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:59.100445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:59.100445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:59.100445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:59.100445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:59.100445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:13:59.100445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:13:59.100445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:13:59.100445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:13:59.100445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:59.100445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:59.100445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:59.100445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:59.100445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:59.100445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:59.101291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:59.101291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:59.101291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:59.101291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:59.101291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:59.101291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:59.101291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:59.101291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:59.101291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:59.101291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:59.101291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:13:59.101291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:13:59.101291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 19:13:59.101291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:59.108649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-c7d3a604-4d18-4f9f-8841-6a9e394aeb1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:59.108649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:59.108649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:59.108649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:59.108649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:59.108649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:59.108649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:59.108649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:59.108649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:59.108649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:59.108649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:59.108649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:59.108649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:13:59.108649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:13:59.108649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:13:59.109561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:13:59.109561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:59.109561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:59.109561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:59.109561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:59.109561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:59.109561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:59.109561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:59.109561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:59.109561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:59.109561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:59.109561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:59.109561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:59.109561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:59.109561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:59.109561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:59.109561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:59.110435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:13:59.110435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:13:59.110435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:13:59.110435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:13:59.110435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:13:59.110435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:13:59.110435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:13:59.110435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:13:59.110435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 404/801] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:59.110911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3a604-4d18-4f9f-8841-6a9e394aeb1b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:59.111462 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 397/801] 10.222.0.17 () {56 vars in 1129 bytes} [Sun Aug 7 19:13:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:59.112579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667d013-d53f-4202-8c6d-0176c5d9a4a6 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:59.140381 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:59.220894 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667d013-d53f-4202-8c6d-0176c5d9a4a6 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:59.238101 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3a604-4d18-4f9f-8841-6a9e394aeb1b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:59.250151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:59.263085 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ec2db-6f07-48b2-b83d-7539e2557f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:59.264133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ec2db-6f07-48b2-b83d-7539e2557f85 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:59.264410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ec2db-6f07-48b2-b83d-7539e2557f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:59.264678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ec2db-6f07-48b2-b83d-7539e2557f85 None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:59.265317 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ec2db-6f07-48b2-b83d-7539e2557f85 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:59.265615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ec2db-6f07-48b2-b83d-7539e2557f85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:59.267021 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ec2db-6f07-48b2-b83d-7539e2557f85 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:59.278354 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 398/802] 10.222.0.17 () {56 vars in 1124 bytes} [Sun Aug 7 19:13:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:59.278354 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ec2db-6f07-48b2-b83d-7539e2557f85 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:59.288591 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2e188-37b0-4d55-9138-aed0f043a71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:59.290061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2e188-37b0-4d55-9138-aed0f043a71e None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:59.290914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2e188-37b0-4d55-9138-aed0f043a71e None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:59.291541 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2e188-37b0-4d55-9138-aed0f043a71e None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:59.292638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2e188-37b0-4d55-9138-aed0f043a71e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:59.293114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2e188-37b0-4d55-9138-aed0f043a71e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:59.295594 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2e188-37b0-4d55-9138-aed0f043a71e None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:59.319602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 405/803] 10.222.0.17 () {56 vars in 1135 bytes} [Sun Aug 7 19:13:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:59.321367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2e188-37b0-4d55-9138-aed0f043a71e None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:59.351933 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:59.393818 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ec2db-6f07-48b2-b83d-7539e2557f85 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:59.443103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8d5ed-39ab-4c17-bde3-09bf7ac99ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:59.444221 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8d5ed-39ab-4c17-bde3-09bf7ac99ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:59.444597 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8d5ed-39ab-4c17-bde3-09bf7ac99ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:59.444961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8d5ed-39ab-4c17-bde3-09bf7ac99ad7 None admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:59.446376 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8d5ed-39ab-4c17-bde3-09bf7ac99ad7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:59.446776 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8d5ed-39ab-4c17-bde3-09bf7ac99ad7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:59.448367 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8d5ed-39ab-4c17-bde3-09bf7ac99ad7 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:59.448554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2e188-37b0-4d55-9138-aed0f043a71e None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:59.463444 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 399/804] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:13:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:59.464658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8d5ed-39ab-4c17-bde3-09bf7ac99ad7 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:59.485164 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce666339-ce4b-4a76-a65d-5db03386e576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:59.486018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce666339-ce4b-4a76-a65d-5db03386e576 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:59.486438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce666339-ce4b-4a76-a65d-5db03386e576 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:59.486844 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce666339-ce4b-4a76-a65d-5db03386e576 None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:59.487614 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce666339-ce4b-4a76-a65d-5db03386e576 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:59.488074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce666339-ce4b-4a76-a65d-5db03386e576 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:59.489597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce666339-ce4b-4a76-a65d-5db03386e576 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:59.499545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 406/805] 10.222.0.17 () {56 vars in 1124 bytes} [Sun Aug 7 19:13:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:59.500691 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce666339-ce4b-4a76-a65d-5db03386e576 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:59.532227 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:59.572136 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8d5ed-39ab-4c17-bde3-09bf7ac99ad7 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:59.601285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:59.607503 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce666339-ce4b-4a76-a65d-5db03386e576 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:59.612260 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82d406-2e12-4fce-b733-76d7047004f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:59.612912 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82d406-2e12-4fce-b733-76d7047004f9 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:59.613190 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82d406-2e12-4fce-b733-76d7047004f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:59.613460 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82d406-2e12-4fce-b733-76d7047004f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:59.625846 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82d406-2e12-4fce-b733-76d7047004f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:59.627324 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82d406-2e12-4fce-b733-76d7047004f9 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:59.641529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-6d82d406-2e12-4fce-b733-76d7047004f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:59.641529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:13:59.641529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:13:59.641529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:13:59.641529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:13:59.641529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:13:59.641529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:13:59.641529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:13:59.641529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:13:59.641529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:13:59.641529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:13:59.641529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:13:59.641529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:13:59.641529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:13:59.641529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:13:59.641529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:13:59.642503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:13:59.642503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:13:59.642503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:13:59.642503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:13:59.642503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:13:59.642503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:13:59.642503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:13:59.642503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:13:59.642503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:13:59.642503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:13:59.642503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:13:59.642503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:13:59.642503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:13:59.642503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:13:59.642503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:13:59.642503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:13:59.642503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:13:59.643395 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:13:59.643395 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:13:59.643395 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:13:59.643395 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 400/806] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:13:59.643395 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82d406-2e12-4fce-b733-76d7047004f9 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:59.649797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf1294d-4193-45f9-81c3-8da1238df790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:59.650486 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1294d-4193-45f9-81c3-8da1238df790 None admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:59.650770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1294d-4193-45f9-81c3-8da1238df790 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:59.651036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf1294d-4193-45f9-81c3-8da1238df790 None admin] PATH_INFO: `/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/61ff29258a214817b8fc15b51fe8b3e6/roles/43aa4a9830054a09baa1f0aecfd125e2` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:59.670209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1294d-4193-45f9-81c3-8da1238df790 None admin] RBAC: Authorizing `identity:create_grant(project_id=37703d50cd7e4d9bbdc797a7a1466b66, user_id=61ff29258a214817b8fc15b51fe8b3e6, role_id=43aa4a9830054a09baa1f0aecfd125e2)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:59.672134 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf1294d-4193-45f9-81c3-8da1238df790 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:59.692855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 407/807] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:13:59 2022] PUT /identity/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/61ff29258a214817b8fc15b51fe8b3e6/roles/43aa4a9830054a09baa1f0aecfd125e2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:13:59.692855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385f2fa-5478-40dd-a128-7a219afb85dc None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:59.692855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385f2fa-5478-40dd-a128-7a219afb85dc None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:59.692855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385f2fa-5478-40dd-a128-7a219afb85dc None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:59.698443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 408/808] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:13:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:13:59.698443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385f2fa-5478-40dd-a128-7a219afb85dc None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:59.783419 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82d406-2e12-4fce-b733-76d7047004f9 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:59.812509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:13:59.831189 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc7d5d7-cbfe-4c59-85ed-34732d7d61e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:59.831917 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7d5d7-cbfe-4c59-85ed-34732d7d61e4 None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:59.832229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7d5d7-cbfe-4c59-85ed-34732d7d61e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:59.832501 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc7d5d7-cbfe-4c59-85ed-34732d7d61e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:59.833217 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7d5d7-cbfe-4c59-85ed-34732d7d61e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:59.834657 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc7d5d7-cbfe-4c59-85ed-34732d7d61e4 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:59.843937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385f2fa-5478-40dd-a128-7a219afb85dc None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:13:59.868426 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 401/809] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:13:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:13:59.870773 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7826952-24e6-4fcf-b9c0-cd32783542bd None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:59.871066 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7826952-24e6-4fcf-b9c0-cd32783542bd None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:59.871344 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7826952-24e6-4fcf-b9c0-cd32783542bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:59.911130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-e7826952-24e6-4fcf-b9c0-cd32783542bd None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:13:59.912095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6183b-17c8-417b-b24a-abe763b7e066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:13:59.912422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e7826952-24e6-4fcf-b9c0-cd32783542bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:13:59.913288 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6183b-17c8-417b-b24a-abe763b7e066 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:13:59.913779 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6183b-17c8-417b-b24a-abe763b7e066 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:13:59.914246 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6183b-17c8-417b-b24a-abe763b7e066 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:13:59.915449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6183b-17c8-417b-b24a-abe763b7e066 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:13:59.916048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6183b-17c8-417b-b24a-abe763b7e066 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:13:59.919018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6183b-17c8-417b-b24a-abe763b7e066 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:13:59.937917 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 409/810] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:13:59.943918 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6183b-17c8-417b-b24a-abe763b7e066 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:13:59.953218 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:00.050235 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 402/811] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:13:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:00.076115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7826952-24e6-4fcf-b9c0-cd32783542bd None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:00.119795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6183b-17c8-417b-b24a-abe763b7e066 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:00.162678 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e44425-fea4-4ae6-9fd4-efe04f073c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:00.163339 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e44425-fea4-4ae6-9fd4-efe04f073c16 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:00.163613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e44425-fea4-4ae6-9fd4-efe04f073c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:00.163885 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e44425-fea4-4ae6-9fd4-efe04f073c16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:00.167875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:00.173281 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e44425-fea4-4ae6-9fd4-efe04f073c16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:00.175218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e44425-fea4-4ae6-9fd4-efe04f073c16 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:00.190755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-65e44425-fea4-4ae6-9fd4-efe04f073c16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:00.190755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:00.190755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:00.190755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:00.190755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:00.190755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:00.190755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:00.190755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:00.190755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:00.190755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:00.190755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:00.190755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:00.190755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:14:00.190755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:14:00.190755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:14:00.191851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:14:00.191851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:00.191851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:00.191851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:00.191851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:00.191851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:00.191851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:00.191851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:00.191851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:00.191851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:00.191851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:00.191851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:00.191851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:00.191851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:00.191851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:00.191851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:00.191851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:00.192766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:14:00.192766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:14:00.192766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:14:00.192766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:14:00.192766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:14:00.192766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:14:00.192766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:00.192766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:00.193583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 410/812] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:13:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:00.199782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e44425-fea4-4ae6-9fd4-efe04f073c16 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:00.203308 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7826952-24e6-4fcf-b9c0-cd32783542bd None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:00.254311 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f14973-812f-407d-bdc0-7458f3e6dd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:00.255462 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f14973-812f-407d-bdc0-7458f3e6dd0f None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:00.255795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f14973-812f-407d-bdc0-7458f3e6dd0f None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:00.256365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f14973-812f-407d-bdc0-7458f3e6dd0f None admin] PATH_INFO: `/v3/services/network` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:00.257021 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f14973-812f-407d-bdc0-7458f3e6dd0f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:00.258916 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f14973-812f-407d-bdc0-7458f3e6dd0f None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:00.271184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-86f14973-812f-407d-bdc0-7458f3e6dd0f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 19:14:00.271184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:00.271184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:00.271184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:00.271184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:00.271184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:00.271184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:00.271184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:00.271184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 19:14:00.271184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:00.271184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:00.271184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:00.271184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:00.271184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:00.271184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:00.271184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:00.271184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:00.272765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:00.272765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:00.272765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:00.272765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:00.272765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:00.272765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:00.272765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:00.272765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:00.272765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:00.272765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:00.272765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:00.272765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:00.272765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:00.272765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:00.272765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:00.272765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:00.274817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:00.274817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:00.274817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:00.274817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:00.274817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:00.274817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:00.274817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:00.274817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:00.274817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:00.274817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:00.274817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:00.274817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:00.274817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 19:14:00.274817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:00.274817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 403/813] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:14:00 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:00.276616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d3069-162a-4460-9d24-318d397803ae None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:00.276616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d3069-162a-4460-9d24-318d397803ae None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:00.276616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d3069-162a-4460-9d24-318d397803ae None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:00.276616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 404/814] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:00.281947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d3069-162a-4460-9d24-318d397803ae None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:00.336974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e44425-fea4-4ae6-9fd4-efe04f073c16 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:00.360409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:00.378303 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0154a175-310a-47d7-8251-0762cebb4f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:00.378930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154a175-310a-47d7-8251-0762cebb4f3f None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:00.379211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154a175-310a-47d7-8251-0762cebb4f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:00.379487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0154a175-310a-47d7-8251-0762cebb4f3f None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:00.380216 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154a175-310a-47d7-8251-0762cebb4f3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:00.380527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154a175-310a-47d7-8251-0762cebb4f3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:00.382049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0154a175-310a-47d7-8251-0762cebb4f3f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:00.387537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d3069-162a-4460-9d24-318d397803ae None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:00.395915 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 411/815] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:14:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:00.397311 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2d065-670c-4e91-b471-4345c702ca2c None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:00.397610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2d065-670c-4e91-b471-4345c702ca2c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:00.397864 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2d065-670c-4e91-b471-4345c702ca2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:00.416555 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-c0b2d065-670c-4e91-b471-4345c702ca2c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:00.417835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c0b2d065-670c-4e91-b471-4345c702ca2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:00.424323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971a531-4bf8-4800-b5a3-c9bce0e8282a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:00.425027 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971a531-4bf8-4800-b5a3-c9bce0e8282a None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:00.425374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971a531-4bf8-4800-b5a3-c9bce0e8282a None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:00.425715 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971a531-4bf8-4800-b5a3-c9bce0e8282a None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:00.426423 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a971a531-4bf8-4800-b5a3-c9bce0e8282a None admin] RBAC: Adding query filter params (name=network) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:00.426796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a971a531-4bf8-4800-b5a3-c9bce0e8282a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:00.428289 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a971a531-4bf8-4800-b5a3-c9bce0e8282a None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:00.434008 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 405/816] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Aug 7 19:14:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:00.434904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971a531-4bf8-4800-b5a3-c9bce0e8282a None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:00.480711 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:00.508321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 412/817] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:00.509574 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2d065-670c-4e91-b471-4345c702ca2c None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:00.544020 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:00.547342 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971a531-4bf8-4800-b5a3-c9bce0e8282a None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:00.587428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a5aa4-d49d-45e3-a72e-922ef89ef394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:00.588161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a5aa4-d49d-45e3-a72e-922ef89ef394 None admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:00.588512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a5aa4-d49d-45e3-a72e-922ef89ef394 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:00.588857 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8a5aa4-d49d-45e3-a72e-922ef89ef394 None admin] PATH_INFO: `/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/db51971158a946c98a8ed716ecc5d72b/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:00.591352 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a5aa4-d49d-45e3-a72e-922ef89ef394 None admin] RBAC: Authorizing `identity:create_grant(project_id=37703d50cd7e4d9bbdc797a7a1466b66, user_id=db51971158a946c98a8ed716ecc5d72b, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:00.592944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8a5aa4-d49d-45e3-a72e-922ef89ef394 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:00.611690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 406/818] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:14:00 2022] PUT /identity/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/db51971158a946c98a8ed716ecc5d72b/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:14:00.612787 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a5aa4-d49d-45e3-a72e-922ef89ef394 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:00.645044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b2d065-670c-4e91-b471-4345c702ca2c None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:00.685536 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c3836-8956-494a-a4a4-707043d897cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:00.686182 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c3836-8956-494a-a4a4-707043d897cc None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:00.686462 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c3836-8956-494a-a4a4-707043d897cc None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:00.686740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c3836-8956-494a-a4a4-707043d897cc None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:00.687479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95c3836-8956-494a-a4a4-707043d897cc None admin] RBAC: Adding query filter params (type=network) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:00.687790 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95c3836-8956-494a-a4a4-707043d897cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:00.689548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95c3836-8956-494a-a4a4-707043d897cc None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:00.695811 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 413/819] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Aug 7 19:14:00 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:00.702385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c3836-8956-494a-a4a4-707043d897cc None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:00.748686 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8a5aa4-d49d-45e3-a72e-922ef89ef394 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:00.762374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:00.839256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a370a-9e41-467c-8529-18e699d6f404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:00.839973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a370a-9e41-467c-8529-18e699d6f404 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:00.840360 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a370a-9e41-467c-8529-18e699d6f404 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:00.840707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a370a-9e41-467c-8529-18e699d6f404 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:00.855819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a370a-9e41-467c-8529-18e699d6f404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:14:00.861535 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c3836-8956-494a-a4a4-707043d897cc None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:00.861786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-6f5a370a-9e41-467c-8529-18e699d6f404 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:14:00.861786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:00.861786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:00.861786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:00.861786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:00.861786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:00.861786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:00.861786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:00.861786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:00.861786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:00.861786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:00.861786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:00.861786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:14:00.861786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:14:00.861786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:14:00.861786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:14:00.862889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:14:00.862889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:14:00.862889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:14:00.862889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:14:00.862889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:00.862889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:00.862889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:00.862889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:00.862889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:00.862889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:00.862889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:00.862889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:00.862889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:00.862889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:00.862889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:00.862889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:00.863920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:00.863920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:00.863920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:00.863920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:00.863920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:14:00.863920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:14:00.863920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:14:00.863920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:14:00.863920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:14:00.863920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:14:00.863920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:14:00.863920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:00.865906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 407/820] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:14:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:00.872238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a370a-9e41-467c-8529-18e699d6f404 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:00.889395 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:00.910091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6e95e-c2b8-460b-b366-1253dd5fb4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:00.910746 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6e95e-c2b8-460b-b366-1253dd5fb4e5 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:00.911028 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6e95e-c2b8-460b-b366-1253dd5fb4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:00.911306 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6e95e-c2b8-460b-b366-1253dd5fb4e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:00.913125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6e95e-c2b8-460b-b366-1253dd5fb4e5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:00.913538 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c6e95e-c2b8-460b-b366-1253dd5fb4e5 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:00.948996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 414/821] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:14:00 2022] POST /identity/v3/endpoints => generated 337 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:14:01.015007 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a370a-9e41-467c-8529-18e699d6f404 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:01.084290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb124d-c524-435a-9789-3a39c46018ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:01.084927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb124d-c524-435a-9789-3a39c46018ec None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:01.085210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb124d-c524-435a-9789-3a39c46018ec None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:01.085484 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb124d-c524-435a-9789-3a39c46018ec None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:01.086171 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabb124d-c524-435a-9789-3a39c46018ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:01.086481 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabb124d-c524-435a-9789-3a39c46018ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:01.087859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabb124d-c524-435a-9789-3a39c46018ec None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:01.100853 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 408/822] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:14:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:01.106670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6e95e-c2b8-460b-b366-1253dd5fb4e5 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:01.269043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c6e95e-c2b8-460b-b366-1253dd5fb4e5 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:01.277420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:01.313993 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626526bc-3112-4812-88da-c6d6a0493d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:01.314993 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626526bc-3112-4812-88da-c6d6a0493d23 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:01.315467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626526bc-3112-4812-88da-c6d6a0493d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:01.315976 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626526bc-3112-4812-88da-c6d6a0493d23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:01.327301 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626526bc-3112-4812-88da-c6d6a0493d23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:01.329113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626526bc-3112-4812-88da-c6d6a0493d23 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:01.344524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-626526bc-3112-4812-88da-c6d6a0493d23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:01.344524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:01.344524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:01.344524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:01.344524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:01.344524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:01.344524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:01.344524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:01.344524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:01.344524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:01.344524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:01.344524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:01.344524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:01.344524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:01.344524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:01.344524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:01.345924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:01.345924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:01.345924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:01.345924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:01.345924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:01.345924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:01.345924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:01.345924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:01.345924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:01.345924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:01.345924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:01.345924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:01.345924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:01.345924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:01.345924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:01.345924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:01.345924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:01.346830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:01.346830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:01.346830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:01.349115 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 415/823] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:01.356086 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb124d-c524-435a-9789-3a39c46018ec None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:01.445382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:01.483187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb124d-c524-435a-9789-3a39c46018ec None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:01.527252 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c505967-0421-4d21-8073-2ae8198bd499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:01.527886 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c505967-0421-4d21-8073-2ae8198bd499 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:01.528210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c505967-0421-4d21-8073-2ae8198bd499 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:01.528479 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c505967-0421-4d21-8073-2ae8198bd499 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:01.529131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c505967-0421-4d21-8073-2ae8198bd499 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:01.529434 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c505967-0421-4d21-8073-2ae8198bd499 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:01.530811 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c505967-0421-4d21-8073-2ae8198bd499 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:01.542995 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 409/824] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:01.548734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626526bc-3112-4812-88da-c6d6a0493d23 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:01.657980 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626526bc-3112-4812-88da-c6d6a0493d23 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:01.699046 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e74f5-0079-429f-a2c6-0cd0c0088cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:01.700226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e74f5-0079-429f-a2c6-0cd0c0088cbf None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:01.700706 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e74f5-0079-429f-a2c6-0cd0c0088cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:01.701172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e74f5-0079-429f-a2c6-0cd0c0088cbf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:01.710068 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e74f5-0079-429f-a2c6-0cd0c0088cbf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:01.711880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e74f5-0079-429f-a2c6-0cd0c0088cbf None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:01.723812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-6a6e74f5-0079-429f-a2c6-0cd0c0088cbf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:14:01.723812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:01.723812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:01.723812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:01.723812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:01.723812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:01.723812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:01.723812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:01.723812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:01.723812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:01.723812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:01.723812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:01.723812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:14:01.723812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:14:01.723812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:14:01.723812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:01.724797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:01.725744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:01.725744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:01.725744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:14:01.725744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:14:01.725744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:14:01.725744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:14:01.725744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:14:01.725744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:14:01.725744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:14:01.725744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:01.728282 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 416/825] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Aug 7 19:14:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:01.736036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c505967-0421-4d21-8073-2ae8198bd499 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:01.834618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c505967-0421-4d21-8073-2ae8198bd499 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:01.864198 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:01.870250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e18cc-1769-469e-8989-253562fbc6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:01.871350 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e18cc-1769-469e-8989-253562fbc6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:01.871958 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e18cc-1769-469e-8989-253562fbc6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:01.872536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e18cc-1769-469e-8989-253562fbc6b6 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:01.873664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e18cc-1769-469e-8989-253562fbc6b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:01.873971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e18cc-1769-469e-8989-253562fbc6b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:01.875418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e18cc-1769-469e-8989-253562fbc6b6 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:01.894259 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 410/826] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Aug 7 19:14:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:01.899661 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e74f5-0079-429f-a2c6-0cd0c0088cbf None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:01.925584 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:01.935045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53d679-462e-446d-97c8-c392141570e0 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:01.935366 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53d679-462e-446d-97c8-c392141570e0 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:01.935612 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b53d679-462e-446d-97c8-c392141570e0 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:01.940115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 411/827] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:14:01.942715 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab91d3b-36ce-48f4-a07d-2bee444da8ea None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:01.943012 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab91d3b-36ce-48f4-a07d-2bee444da8ea None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:01.943288 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab91d3b-36ce-48f4-a07d-2bee444da8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:01.964055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-5ab91d3b-36ce-48f4-a07d-2bee444da8ea None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:01.966409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5ab91d3b-36ce-48f4-a07d-2bee444da8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:02.025418 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e74f5-0079-429f-a2c6-0cd0c0088cbf None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:02.064014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 412/828] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:02.066821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b43a6-527f-4d5f-9d00-c38341abaa38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:02.067447 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b43a6-527f-4d5f-9d00-c38341abaa38 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:02.067727 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b43a6-527f-4d5f-9d00-c38341abaa38 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:02.068018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b43a6-527f-4d5f-9d00-c38341abaa38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:02.082806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab91d3b-36ce-48f4-a07d-2bee444da8ea None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:02.145396 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b43a6-527f-4d5f-9d00-c38341abaa38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:02.147266 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b43a6-527f-4d5f-9d00-c38341abaa38 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:02.168554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:02.215094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab91d3b-36ce-48f4-a07d-2bee444da8ea None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:02.234842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:02.254090 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72542b-a716-4fb3-8cdd-bbad014cb024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:02.255137 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72542b-a716-4fb3-8cdd-bbad014cb024 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:02.255616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72542b-a716-4fb3-8cdd-bbad014cb024 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:02.256118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72542b-a716-4fb3-8cdd-bbad014cb024 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:02.267316 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72542b-a716-4fb3-8cdd-bbad014cb024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:14:02.272497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-1d72542b-a716-4fb3-8cdd-bbad014cb024 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:14:02.272497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:02.272497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:02.272497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:02.272497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:02.272497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:02.272497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:02.272497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:02.272497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:02.272497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:02.272497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:02.272497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:02.272497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:14:02.272497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:14:02.272497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:14:02.272497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:14:02.273424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:14:02.273424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:14:02.273424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:14:02.273424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:14:02.273424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:02.273424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:02.273424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:02.273424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:02.273424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:02.273424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:02.273424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:02.273424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:02.273424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:02.273424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:02.273424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:02.273424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:02.274242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:02.274242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:02.274242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:02.274242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:02.274242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:14:02.274242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:14:02.274242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:14:02.274242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:14:02.274242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:14:02.274242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:14:02.274242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:14:02.274242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:02.282234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 413/829] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:14:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:02.283836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72542b-a716-4fb3-8cdd-bbad014cb024 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:02.284851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 417/830] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:14:01 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:14:02.293276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b43a6-527f-4d5f-9d00-c38341abaa38 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:02.392103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72542b-a716-4fb3-8cdd-bbad014cb024 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:02.424290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b43a6-527f-4d5f-9d00-c38341abaa38 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:02.427611 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327784c-a6c8-467b-8a22-7954ef12cabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:02.428277 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327784c-a6c8-467b-8a22-7954ef12cabf None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:02.428557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327784c-a6c8-467b-8a22-7954ef12cabf None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:02.428830 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327784c-a6c8-467b-8a22-7954ef12cabf None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:02.429507 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327784c-a6c8-467b-8a22-7954ef12cabf None admin] RBAC: Adding query filter params (name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:02.429820 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327784c-a6c8-467b-8a22-7954ef12cabf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:02.431205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327784c-a6c8-467b-8a22-7954ef12cabf None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:02.440634 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 414/831] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:14:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:02.444142 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96883855-dcee-47fe-8420-55a3022f63d3 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:02.444142 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96883855-dcee-47fe-8420-55a3022f63d3 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:02.444142 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96883855-dcee-47fe-8420-55a3022f63d3 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:02.444142 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 415/832] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:02.448973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96883855-dcee-47fe-8420-55a3022f63d3 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:02.452534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:02.472941 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dcd628-0652-4313-9e0e-3a9a74603f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:02.473505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcd628-0652-4313-9e0e-3a9a74603f5d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:02.473784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcd628-0652-4313-9e0e-3a9a74603f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:02.474056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcd628-0652-4313-9e0e-3a9a74603f5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:02.484993 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dcd628-0652-4313-9e0e-3a9a74603f5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:02.486483 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dcd628-0652-4313-9e0e-3a9a74603f5d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:02.499946 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:02.504059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-56dcd628-0652-4313-9e0e-3a9a74603f5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:02.504059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:02.504059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:02.504059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:02.504059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:02.504059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:02.504059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:02.504059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:02.504059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:02.504059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:02.504059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:02.504059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:02.504059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:02.504059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:02.504059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:02.504059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:02.512645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:02.512645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:02.512645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:02.512645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:02.512645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:02.512645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:02.512645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:02.512645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:02.512645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:02.512645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:02.512645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:02.512645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:02.512645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:02.512645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:02.512645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:02.512645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:02.512645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:02.513540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:02.513540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:02.513540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:02.513540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 418/833] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:02.513540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc04f6-c885-48db-9536-66ab527e6a41 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:02.513540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc04f6-c885-48db-9536-66ab527e6a41 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:02.513540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc04f6-c885-48db-9536-66ab527e6a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:02.526949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-fcdc04f6-c885-48db-9536-66ab527e6a41 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:02.528187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-fcdc04f6-c885-48db-9536-66ab527e6a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:02.577498 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96883855-dcee-47fe-8420-55a3022f63d3 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:02.600630 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 419/834] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:02.601527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc04f6-c885-48db-9536-66ab527e6a41 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:02.627047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb225a8-8a66-4d71-b5c1-811230abfb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:02.627866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb225a8-8a66-4d71-b5c1-811230abfb92 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:02.628164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb225a8-8a66-4d71-b5c1-811230abfb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:02.628505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb225a8-8a66-4d71-b5c1-811230abfb92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:02.639941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb225a8-8a66-4d71-b5c1-811230abfb92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:02.641411 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb225a8-8a66-4d71-b5c1-811230abfb92 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:02.658087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-bdb225a8-8a66-4d71-b5c1-811230abfb92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:02.658087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:02.658087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:02.658087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:02.658087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:02.658087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:02.658087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:02.658087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:02.658087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:02.658087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:02.658087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:02.658087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:02.658087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:02.658087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:02.658087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:02.658087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:02.659045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:02.659045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:02.659045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:02.659045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:02.659045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:02.659045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:02.659045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:02.659045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:02.659045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:02.659045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:02.659045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:02.659045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:02.659045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:02.659045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:02.659045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:02.659045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:02.659045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:02.659922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:02.659922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:02.659922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:02.661385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 416/835] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:02.662429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb225a8-8a66-4d71-b5c1-811230abfb92 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:02.708461 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:02.711104 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc04f6-c885-48db-9536-66ab527e6a41 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:02.753887 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d088c87-550b-4a16-b54a-f31a75a00ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:02.754582 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d088c87-550b-4a16-b54a-f31a75a00ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:02.754917 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d088c87-550b-4a16-b54a-f31a75a00ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:02.755523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d088c87-550b-4a16-b54a-f31a75a00ea4 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:02.755957 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d088c87-550b-4a16-b54a-f31a75a00ea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:02.756347 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d088c87-550b-4a16-b54a-f31a75a00ea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:02.758362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d088c87-550b-4a16-b54a-f31a75a00ea4 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:02.767452 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:02.775311 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 420/836] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:02.776185 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d088c87-550b-4a16-b54a-f31a75a00ea4 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:02.777904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb225a8-8a66-4d71-b5c1-811230abfb92 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:02.823297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ecf11-217b-4892-9f11-2af80a92aa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:02.824069 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ecf11-217b-4892-9f11-2af80a92aa79 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:02.824390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ecf11-217b-4892-9f11-2af80a92aa79 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:02.825055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ecf11-217b-4892-9f11-2af80a92aa79 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:02.825663 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ecf11-217b-4892-9f11-2af80a92aa79 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:02.828951 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ecf11-217b-4892-9f11-2af80a92aa79 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:02.838998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-f74ecf11-217b-4892-9f11-2af80a92aa79 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 19:14:02.838998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:02.838998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:02.838998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:02.838998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:02.838998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:02.838998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:02.838998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:02.838998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 19:14:02.838998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:02.838998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:02.838998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:02.838998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:02.838998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:02.838998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:02.838998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:02.838998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:02.840216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:02.840216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:02.840216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:02.840216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:02.840216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:02.840216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:02.840216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:02.840216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:02.840216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:02.840216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:02.840216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:02.840216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:02.840216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:02.840216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:02.840216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:02.840216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:02.841210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:02.841210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:02.841210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:02.841210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:02.841210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:02.841210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:02.841210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:02.841210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:02.841210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:02.841210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:02.841210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:02.841210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:02.841210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 19:14:02.841210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:02.845251 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 417/837] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:14:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:02.846263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ecf11-217b-4892-9f11-2af80a92aa79 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:02.897768 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d088c87-550b-4a16-b54a-f31a75a00ea4 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:02.936377 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb97128-c245-4d30-9f36-e033539ea763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:02.937081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb97128-c245-4d30-9f36-e033539ea763 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:02.937462 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb97128-c245-4d30-9f36-e033539ea763 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:02.937804 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb97128-c245-4d30-9f36-e033539ea763 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:02.938534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb97128-c245-4d30-9f36-e033539ea763 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:02.938948 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb97128-c245-4d30-9f36-e033539ea763 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:02.940426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb97128-c245-4d30-9f36-e033539ea763 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:02.957570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 421/838] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:02.957570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb97128-c245-4d30-9f36-e033539ea763 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:02.959585 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ecf11-217b-4892-9f11-2af80a92aa79 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:02.982903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:03.000578 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da40824-b46c-4600-b31b-90901dd8e09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:03.001208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da40824-b46c-4600-b31b-90901dd8e09d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:03.001495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da40824-b46c-4600-b31b-90901dd8e09d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:03.001766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da40824-b46c-4600-b31b-90901dd8e09d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:03.009034 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da40824-b46c-4600-b31b-90901dd8e09d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:03.010432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da40824-b46c-4600-b31b-90901dd8e09d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:03.021953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-1da40824-b46c-4600-b31b-90901dd8e09d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:03.021953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:03.021953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:03.021953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:03.021953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:03.021953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:03.021953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:03.021953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:03.021953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:03.021953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:03.021953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:03.021953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:03.021953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:14:03.021953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:14:03.021953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:14:03.023044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:14:03.023044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:03.023044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:03.023044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:03.023044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:03.023044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:03.023044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:03.023044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:03.023044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:03.023044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:03.023044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:03.023044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:03.023044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:03.023044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:03.023044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:03.023044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:03.023044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:03.023927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:14:03.023927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:14:03.023927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:14:03.023927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:14:03.023927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:14:03.023927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:14:03.023927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:03.023927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:03.023927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 418/839] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:14:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:03.024440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae225b96-e2c7-443c-add5-95a145830542 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:03.024513 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae225b96-e2c7-443c-add5-95a145830542 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:03.024812 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae225b96-e2c7-443c-add5-95a145830542 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:03.026001 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 419/840] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:03.026691 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae225b96-e2c7-443c-add5-95a145830542 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:03.046596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:03.073891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb97128-c245-4d30-9f36-e033539ea763 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:03.112563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd2cfd-44a9-464e-b2a3-8a48593f9cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:03.113334 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd2cfd-44a9-464e-b2a3-8a48593f9cec None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:03.113740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd2cfd-44a9-464e-b2a3-8a48593f9cec None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:03.114147 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd2cfd-44a9-464e-b2a3-8a48593f9cec None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:03.114897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd2cfd-44a9-464e-b2a3-8a48593f9cec None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:03.115302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd2cfd-44a9-464e-b2a3-8a48593f9cec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:03.116997 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbd2cfd-44a9-464e-b2a3-8a48593f9cec None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:03.122313 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 422/841] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:14:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:03.123106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd2cfd-44a9-464e-b2a3-8a48593f9cec None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:03.142553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae225b96-e2c7-443c-add5-95a145830542 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:03.178145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948924dc-f561-40d4-b58d-79ef24b0e7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:03.178763 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948924dc-f561-40d4-b58d-79ef24b0e7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:03.179045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948924dc-f561-40d4-b58d-79ef24b0e7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:03.179310 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948924dc-f561-40d4-b58d-79ef24b0e7e0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:03.186895 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948924dc-f561-40d4-b58d-79ef24b0e7e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:03.188368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948924dc-f561-40d4-b58d-79ef24b0e7e0 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:03.199758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-948924dc-f561-40d4-b58d-79ef24b0e7e0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 19:14:03.199758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:03.199758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:03.199758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:03.199758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:03.199758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:03.199758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:03.199758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:03.199758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:03.199758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:03.199758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:03.199758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:03.199758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:14:03.199758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:14:03.199758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:14:03.199758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:03.200962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:03.202438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:03.202438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:03.202438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:14:03.202438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:14:03.202438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:14:03.202438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:14:03.202438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:14:03.202438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:14:03.202438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 19:14:03.202438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:03.202438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 420/842] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Aug 7 19:14:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:03.205877 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c2c57-d8d3-402e-9208-178656b38d42 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:03.206145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c2c57-d8d3-402e-9208-178656b38d42 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:03.206427 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278c2c57-d8d3-402e-9208-178656b38d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:03.212731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:03.224369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-278c2c57-d8d3-402e-9208-178656b38d42 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:03.225579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-278c2c57-d8d3-402e-9208-178656b38d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:03.230498 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd2cfd-44a9-464e-b2a3-8a48593f9cec None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:03.265213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a8135-fecc-4e47-b42c-f56933004016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:03.265749 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a8135-fecc-4e47-b42c-f56933004016 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:03.266024 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a8135-fecc-4e47-b42c-f56933004016 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:03.266288 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a8135-fecc-4e47-b42c-f56933004016 None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:03.266969 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a8135-fecc-4e47-b42c-f56933004016 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:03.267259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a8135-fecc-4e47-b42c-f56933004016 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:03.268724 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a8135-fecc-4e47-b42c-f56933004016 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:03.280734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 423/843] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:14:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:03.281535 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a8135-fecc-4e47-b42c-f56933004016 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:03.290494 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 421/844] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:03 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:03.291595 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c2c57-d8d3-402e-9208-178656b38d42 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:03.300612 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:03.388628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a8135-fecc-4e47-b42c-f56933004016 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:03.397344 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278c2c57-d8d3-402e-9208-178656b38d42 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:03.423983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5853297-885b-4c0f-9d5e-0063fd192fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:03.424663 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5853297-885b-4c0f-9d5e-0063fd192fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:03.425198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5853297-885b-4c0f-9d5e-0063fd192fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:03.425198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5853297-885b-4c0f-9d5e-0063fd192fe9 None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:03.425813 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5853297-885b-4c0f-9d5e-0063fd192fe9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:03.426380 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5853297-885b-4c0f-9d5e-0063fd192fe9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:03.427492 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5853297-885b-4c0f-9d5e-0063fd192fe9 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:03.432517 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 424/845] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:14:03 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:03.434148 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5853297-885b-4c0f-9d5e-0063fd192fe9 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:03.434413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80f18e-812c-40fb-af12-440b1fa9a4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:03.435101 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80f18e-812c-40fb-af12-440b1fa9a4fa None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:03.435380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80f18e-812c-40fb-af12-440b1fa9a4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:03.435656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80f18e-812c-40fb-af12-440b1fa9a4fa None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:03.436412 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80f18e-812c-40fb-af12-440b1fa9a4fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:03.436724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80f18e-812c-40fb-af12-440b1fa9a4fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:03.438149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80f18e-812c-40fb-af12-440b1fa9a4fa None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:03.455822 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:03.459342 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 422/846] 10.222.0.17 () {56 vars in 1149 bytes} [Sun Aug 7 19:14:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:03.460444 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80f18e-812c-40fb-af12-440b1fa9a4fa None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:03.538987 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:03.575701 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5853297-885b-4c0f-9d5e-0063fd192fe9 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:03.580052 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80f18e-812c-40fb-af12-440b1fa9a4fa None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:03.621314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c58cb-efdf-481d-b0bc-277e8d46a107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:03.621939 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c58cb-efdf-481d-b0bc-277e8d46a107 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:03.622224 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c58cb-efdf-481d-b0bc-277e8d46a107 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:03.622495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c58cb-efdf-481d-b0bc-277e8d46a107 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:03.623084 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4baa26-ae5f-4aca-97da-70b258705eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:03.623281 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c58cb-efdf-481d-b0bc-277e8d46a107 None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=a4946aec299f45bd8402cff3159d04d7) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:03.623532 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c58cb-efdf-481d-b0bc-277e8d46a107 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:03.623726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4baa26-ae5f-4aca-97da-70b258705eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:03.624025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4baa26-ae5f-4aca-97da-70b258705eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:03.624299 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4baa26-ae5f-4aca-97da-70b258705eb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:03.625012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c58cb-efdf-481d-b0bc-277e8d46a107 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:03.631042 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 425/847] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:14:03 2022] GET /identity/v3/role_assignments?user.id=a4946aec299f45bd8402cff3159d04d7&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=4da42f08d107448fb877b42b61bc74ea => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:03.631935 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c58cb-efdf-481d-b0bc-277e8d46a107 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:03.646035 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4baa26-ae5f-4aca-97da-70b258705eb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:14:03.651608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-ed4baa26-ae5f-4aca-97da-70b258705eb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:14:03.651608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:03.651608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:03.651608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:03.651608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:03.651608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:03.651608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:03.651608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:03.651608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:03.651608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:03.651608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:03.651608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:03.651608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:14:03.651608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:14:03.651608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:14:03.651608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:14:03.653062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:14:03.653062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:14:03.653062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:14:03.653062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:14:03.653062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:03.653062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:03.653062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:03.653062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:03.653062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:03.653062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:03.653062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:03.653062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:03.653062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:03.653062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:03.653062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:03.653062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:03.653990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:03.653990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:03.653990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:03.653990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:03.653990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:14:03.653990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:14:03.653990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:14:03.653990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:14:03.653990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:14:03.653990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:14:03.653990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:14:03.653990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:03.654628 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 423/848] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:14:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:03.660784 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4baa26-ae5f-4aca-97da-70b258705eb8 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:03.732044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:03.773833 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c58cb-efdf-481d-b0bc-277e8d46a107 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:03.793036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4baa26-ae5f-4aca-97da-70b258705eb8 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:03.824699 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462f8d7-c804-42de-bc3d-a4cb64a1546b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:03.825331 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462f8d7-c804-42de-bc3d-a4cb64a1546b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:03.825611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462f8d7-c804-42de-bc3d-a4cb64a1546b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:03.825884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462f8d7-c804-42de-bc3d-a4cb64a1546b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:03.828982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:03.844865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2a267-4bca-489c-9ce6-db3d4f24885f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:03.845493 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2a267-4bca-489c-9ce6-db3d4f24885f None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:03.845771 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2a267-4bca-489c-9ce6-db3d4f24885f None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:03.846047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2a267-4bca-489c-9ce6-db3d4f24885f None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:03.846718 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2a267-4bca-489c-9ce6-db3d4f24885f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:03.847016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2a267-4bca-489c-9ce6-db3d4f24885f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:03.848675 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2a267-4bca-489c-9ce6-db3d4f24885f None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:03.858081 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 424/849] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:14:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:03.864601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2a267-4bca-489c-9ce6-db3d4f24885f None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:03.924620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462f8d7-c804-42de-bc3d-a4cb64a1546b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:03.928352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462f8d7-c804-42de-bc3d-a4cb64a1546b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:04.000804 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2a267-4bca-489c-9ce6-db3d4f24885f None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:04.032804 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:04.051896 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4acf3-3290-4b3d-b314-9aa0e8330f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:04.052489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4acf3-3290-4b3d-b314-9aa0e8330f39 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:04.052877 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4acf3-3290-4b3d-b314-9aa0e8330f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:04.053150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4acf3-3290-4b3d-b314-9aa0e8330f39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:04.065231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4acf3-3290-4b3d-b314-9aa0e8330f39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:04.066690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4acf3-3290-4b3d-b314-9aa0e8330f39 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:04.078838 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 426/850] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:14:03 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:14:04.083574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-06f4acf3-3290-4b3d-b314-9aa0e8330f39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:04.083574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:04.083574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:04.083574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:04.083574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:04.083574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:04.083574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:04.083574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:04.083574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:04.083574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:04.083574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:04.083574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:04.083574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:04.083574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:04.083574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:04.083574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:04.084842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:04.084842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:04.084842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:04.084842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:04.084842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:04.084842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:04.084842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:04.084842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:04.084842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:04.084842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:04.084842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:04.084842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:04.084842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:04.084842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:04.084842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:04.084842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:04.084842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:04.085727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:04.085727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:04.085727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:04.085897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462f8d7-c804-42de-bc3d-a4cb64a1546b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:04.088157 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 425/851] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:04.093114 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4acf3-3290-4b3d-b314-9aa0e8330f39 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:04.147069 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:04.217121 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462f8d7-c804-42de-bc3d-a4cb64a1546b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:04.229244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4acf3-3290-4b3d-b314-9aa0e8330f39 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:04.270680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428caac0-3e44-4dc1-8d68-a6ee81434703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:04.271624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428caac0-3e44-4dc1-8d68-a6ee81434703 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:04.271901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428caac0-3e44-4dc1-8d68-a6ee81434703 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:04.272198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428caac0-3e44-4dc1-8d68-a6ee81434703 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:04.276798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ca4f6-95f9-4308-8de1-e9a40f70e178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:04.277677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ca4f6-95f9-4308-8de1-e9a40f70e178 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:04.278157 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ca4f6-95f9-4308-8de1-e9a40f70e178 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:04.278634 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ca4f6-95f9-4308-8de1-e9a40f70e178 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:04.280053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ca4f6-95f9-4308-8de1-e9a40f70e178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:04.280568 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ca4f6-95f9-4308-8de1-e9a40f70e178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:04.282890 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ca4f6-95f9-4308-8de1-e9a40f70e178 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:04.283002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428caac0-3e44-4dc1-8d68-a6ee81434703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:04.284395 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428caac0-3e44-4dc1-8d68-a6ee81434703 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:04.302362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-428caac0-3e44-4dc1-8d68-a6ee81434703 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:04.302362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:04.302362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:04.302362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:04.302362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:04.302362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:04.302362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:04.302362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:04.302362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:04.302362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:04.302362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:04.302362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:04.302362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:04.302362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:04.302362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:04.302362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:04.303324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:04.303324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:04.303324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:04.303324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:04.303324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:04.303324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:04.303324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:04.303324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:04.303324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:04.303324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:04.303324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:04.303324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:04.303324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:04.303324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:04.303324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:04.303324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:04.303324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:04.304362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:04.304362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:04.304362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:04.305346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 426/853] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:04.306552 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ca4f6-95f9-4308-8de1-e9a40f70e178 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:04.308204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 427/853] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:04.309239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428caac0-3e44-4dc1-8d68-a6ee81434703 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:04.347974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:04.425048 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:04.431169 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ca4f6-95f9-4308-8de1-e9a40f70e178 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:04.455238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428caac0-3e44-4dc1-8d68-a6ee81434703 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:04.471083 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb782e2-855a-49dd-b01d-3dd577a3b8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:04.472145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb782e2-855a-49dd-b01d-3dd577a3b8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:04.472636 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb782e2-855a-49dd-b01d-3dd577a3b8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:04.473131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb782e2-855a-49dd-b01d-3dd577a3b8a0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:04.484909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb782e2-855a-49dd-b01d-3dd577a3b8a0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:04.488019 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb782e2-855a-49dd-b01d-3dd577a3b8a0 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:04.500645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3a3ba-bd76-4d8b-9bbc-4b8ca280e30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:04.501262 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3a3ba-bd76-4d8b-9bbc-4b8ca280e30d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:04.501534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3a3ba-bd76-4d8b-9bbc-4b8ca280e30d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:04.501801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3a3ba-bd76-4d8b-9bbc-4b8ca280e30d None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:04.502447 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3a3ba-bd76-4d8b-9bbc-4b8ca280e30d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:04.502743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3a3ba-bd76-4d8b-9bbc-4b8ca280e30d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:04.504157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3a3ba-bd76-4d8b-9bbc-4b8ca280e30d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:04.505030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-5eb782e2-855a-49dd-b01d-3dd577a3b8a0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 19:14:04.505030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:04.505030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:04.505030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:04.505030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:04.505030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:04.505030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:04.505030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:04.505030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:04.505030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:04.505030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:04.505030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:04.505030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:14:04.505030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:14:04.505030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:14:04.505030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:04.506627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:04.509120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:04.509120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:04.509120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:14:04.509120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:14:04.509120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:14:04.509120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:14:04.509120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:14:04.509120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:14:04.509120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 19:14:04.509120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:04.519338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 427/855] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:14:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:04.521299 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb782e2-855a-49dd-b01d-3dd577a3b8a0 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:04.521907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 428/855] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:04.522832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3a3ba-bd76-4d8b-9bbc-4b8ca280e30d None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:04.644643 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:04.654416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb782e2-855a-49dd-b01d-3dd577a3b8a0 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:04.675627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3a3ba-bd76-4d8b-9bbc-4b8ca280e30d None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:04.714038 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae7e9e-d93b-4392-b685-7e5d9c9a21fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:04.715587 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae7e9e-d93b-4392-b685-7e5d9c9a21fe None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:04.716334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae7e9e-d93b-4392-b685-7e5d9c9a21fe None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:04.717077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae7e9e-d93b-4392-b685-7e5d9c9a21fe None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:04.718715 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae7e9e-d93b-4392-b685-7e5d9c9a21fe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:04.719493 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae7e9e-d93b-4392-b685-7e5d9c9a21fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:04.722373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ae7e9e-d93b-4392-b685-7e5d9c9a21fe None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:04.742110 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71e6fc-75d5-4f78-85ad-cd1b1aa3e785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:04.742760 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71e6fc-75d5-4f78-85ad-cd1b1aa3e785 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:04.743033 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71e6fc-75d5-4f78-85ad-cd1b1aa3e785 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:04.743301 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71e6fc-75d5-4f78-85ad-cd1b1aa3e785 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:04.752027 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db71e6fc-75d5-4f78-85ad-cd1b1aa3e785 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:04.752612 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 428/856] 10.222.0.17 () {56 vars in 1135 bytes} [Sun Aug 7 19:14:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:04.753543 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db71e6fc-75d5-4f78-85ad-cd1b1aa3e785 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:04.754828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae7e9e-d93b-4392-b685-7e5d9c9a21fe None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:04.768449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-db71e6fc-75d5-4f78-85ad-cd1b1aa3e785 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:04.768449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:04.768449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:04.768449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:04.768449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:04.768449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:04.768449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:04.768449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:04.768449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:04.768449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:04.768449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:04.768449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:04.768449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:14:04.768449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:14:04.768449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:14:04.769476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:14:04.769476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:04.769476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:04.769476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:04.769476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:04.769476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:04.769476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:04.769476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:04.769476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:04.769476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:04.769476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:04.769476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:04.769476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:04.769476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:04.769476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:04.769476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:04.769476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:04.770389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:14:04.770389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:14:04.770389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:14:04.770389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:14:04.770389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:14:04.770389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:14:04.770389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:04.770389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:04.772104 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 429/857] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:14:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:04.775761 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71e6fc-75d5-4f78-85ad-cd1b1aa3e785 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:04.782815 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:04.888497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae7e9e-d93b-4392-b685-7e5d9c9a21fe None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:04.912386 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71e6fc-75d5-4f78-85ad-cd1b1aa3e785 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:04.930422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7dd19-cd40-46ba-b5f0-2f1908018d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:04.931051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7dd19-cd40-46ba-b5f0-2f1908018d6b None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:04.931330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7dd19-cd40-46ba-b5f0-2f1908018d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:04.931600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7dd19-cd40-46ba-b5f0-2f1908018d6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:04.960384 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb3682-654b-445b-be30-918c2d8f536b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:04.961030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb3682-654b-445b-be30-918c2d8f536b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:04.961313 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb3682-654b-445b-be30-918c2d8f536b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:04.961593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb3682-654b-445b-be30-918c2d8f536b None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:04.962290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb3682-654b-445b-be30-918c2d8f536b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:04.962596 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb3682-654b-445b-be30-918c2d8f536b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:04.964407 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb3682-654b-445b-be30-918c2d8f536b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:04.984263 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 430/858] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:14:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:04.988677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb3682-654b-445b-be30-918c2d8f536b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:04.994509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:05.022036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7dd19-cd40-46ba-b5f0-2f1908018d6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:05.026246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7dd19-cd40-46ba-b5f0-2f1908018d6b None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:05.046190 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:05.136281 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb3682-654b-445b-be30-918c2d8f536b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:05.140755 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 429/859] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:14:04 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:14:05.146048 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7dd19-cd40-46ba-b5f0-2f1908018d6b None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:05.179221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591fb07-d1aa-49d6-ac50-689c2b0f13c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:05.180392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591fb07-d1aa-49d6-ac50-689c2b0f13c4 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:05.181673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591fb07-d1aa-49d6-ac50-689c2b0f13c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:05.182512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591fb07-d1aa-49d6-ac50-689c2b0f13c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:05.183538 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591fb07-d1aa-49d6-ac50-689c2b0f13c4 None admin] RBAC: Adding query filter params (role.id=43aa4a9830054a09baa1f0aecfd125e2, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=61ff29258a214817b8fc15b51fe8b3e6) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:05.184094 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591fb07-d1aa-49d6-ac50-689c2b0f13c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:05.185781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3591fb07-d1aa-49d6-ac50-689c2b0f13c4 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:05.200230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 431/860] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:14:04 2022] GET /identity/v3/role_assignments?user.id=61ff29258a214817b8fc15b51fe8b3e6&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=43aa4a9830054a09baa1f0aecfd125e2 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:05.259627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7dd19-cd40-46ba-b5f0-2f1908018d6b None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:05.279661 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:05.300386 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11bd74d-7902-48a4-8857-294909abdf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:05.301049 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bd74d-7902-48a4-8857-294909abdf4c None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:05.301331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bd74d-7902-48a4-8857-294909abdf4c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:05.301604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11bd74d-7902-48a4-8857-294909abdf4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:05.318489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11bd74d-7902-48a4-8857-294909abdf4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:05.320415 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11bd74d-7902-48a4-8857-294909abdf4c None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:05.337270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-d11bd74d-7902-48a4-8857-294909abdf4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:05.337270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:05.337270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:05.337270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:05.337270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:05.337270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:05.337270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:05.337270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:05.337270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:05.337270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:05.337270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:05.337270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:05.337270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:05.337270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:05.337270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:05.337270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:05.338399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:05.338399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:05.338399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:05.338399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:05.338399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:05.338399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:05.338399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:05.338399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:05.338399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:05.338399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:05.338399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:05.338399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:05.338399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:05.338399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:05.338399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:05.338399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:05.338399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:05.339532 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:05.339532 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:05.339532 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:05.341862 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 430/861] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:05.348041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591fb07-d1aa-49d6-ac50-689c2b0f13c4 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:05.423508 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:05.462265 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3591fb07-d1aa-49d6-ac50-689c2b0f13c4 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:05.498069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5d8f0-84d8-4391-9b0c-a2261a8f1eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:05.498432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90e711-eb99-4e3b-a165-0fbb510648a7 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:05.498669 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90e711-eb99-4e3b-a165-0fbb510648a7 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:05.498768 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5d8f0-84d8-4391-9b0c-a2261a8f1eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:05.498942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90e711-eb99-4e3b-a165-0fbb510648a7 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:05.499038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5d8f0-84d8-4391-9b0c-a2261a8f1eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:05.499292 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5d8f0-84d8-4391-9b0c-a2261a8f1eb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:05.500046 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5d8f0-84d8-4391-9b0c-a2261a8f1eb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:05.500339 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5d8f0-84d8-4391-9b0c-a2261a8f1eb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:05.500629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 431/862] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:05.501744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5d8f0-84d8-4391-9b0c-a2261a8f1eb8 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:05.506126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1ea3d-ceaa-415b-9fa7-211cf2e38bbc None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:05.506472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1ea3d-ceaa-415b-9fa7-211cf2e38bbc None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:05.506755 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1ea3d-ceaa-415b-9fa7-211cf2e38bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:05.518529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 432/863] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:05.525271 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5d8f0-84d8-4391-9b0c-a2261a8f1eb8 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:05.533595 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-e4f1ea3d-ceaa-415b-9fa7-211cf2e38bbc None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:05.534928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e4f1ea3d-ceaa-415b-9fa7-211cf2e38bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:05.612681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 432/864] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:05 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 19:14:05.634966 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1ea3d-ceaa-415b-9fa7-211cf2e38bbc None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:05.656752 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5d8f0-84d8-4391-9b0c-a2261a8f1eb8 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:05.689549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:05.699888 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c2ed4-546d-45fd-ac9e-b9d0467d4aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:05.701032 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c2ed4-546d-45fd-ac9e-b9d0467d4aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:05.701769 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c2ed4-546d-45fd-ac9e-b9d0467d4aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:05.702056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1c2ed4-546d-45fd-ac9e-b9d0467d4aa5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:05.712923 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c2ed4-546d-45fd-ac9e-b9d0467d4aa5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:05.714385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1c2ed4-546d-45fd-ac9e-b9d0467d4aa5 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:05.731520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-6c1c2ed4-546d-45fd-ac9e-b9d0467d4aa5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:05.731520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:05.731520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:05.731520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:05.731520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:05.731520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:05.731520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:05.731520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:05.731520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:05.731520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:05.731520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:05.731520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:05.731520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:14:05.731520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:14:05.731520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:14:05.732512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:14:05.732512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:05.732512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:05.732512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:05.732512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:05.732512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:05.732512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:05.732512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:05.732512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:05.732512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:05.732512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:05.732512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:05.732512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:05.732512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:05.732512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:05.732512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:05.732512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:05.733412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:14:05.733412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:14:05.733412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:14:05.733412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:14:05.733412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:14:05.733412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:14:05.733412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:05.733412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:05.733842 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 433/865] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:14:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:05.739732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c2ed4-546d-45fd-ac9e-b9d0467d4aa5 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:05.746865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:05.755052 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1ea3d-ceaa-415b-9fa7-211cf2e38bbc None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:05.793241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e3dd8-5b6b-4266-b11c-6bd49965635d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:05.793505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e3dd8-5b6b-4266-b11c-6bd49965635d None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:05.793731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e3dd8-5b6b-4266-b11c-6bd49965635d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:05.793977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e3dd8-5b6b-4266-b11c-6bd49965635d None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:05.794643 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e3dd8-5b6b-4266-b11c-6bd49965635d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:05.796064 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629e3dd8-5b6b-4266-b11c-6bd49965635d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:05.824244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 433/866] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Aug 7 19:14:05 2022] POST /identity/v3/services => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:14:05.826524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b9417-3282-402e-8a47-84525202b79a None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:05.826865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b9417-3282-402e-8a47-84525202b79a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:05.827149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b9417-3282-402e-8a47-84525202b79a None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:05.833663 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 434/867] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:05.836478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb588f94-33be-4e6e-a5c8-0d7098966add None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:05.836852 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb588f94-33be-4e6e-a5c8-0d7098966add None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:05.837202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb588f94-33be-4e6e-a5c8-0d7098966add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:05.862668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-cb588f94-33be-4e6e-a5c8-0d7098966add None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:05.863915 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-cb588f94-33be-4e6e-a5c8-0d7098966add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:05.879641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1c2ed4-546d-45fd-ac9e-b9d0467d4aa5 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:05.948331 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9699ec-fa23-4402-9751-80dd38c89927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:05.952632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9699ec-fa23-4402-9751-80dd38c89927 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:05.952632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9699ec-fa23-4402-9751-80dd38c89927 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:05.952632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9699ec-fa23-4402-9751-80dd38c89927 None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:05.952632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9699ec-fa23-4402-9751-80dd38c89927 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:05.952632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9699ec-fa23-4402-9751-80dd38c89927 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:05.953972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9699ec-fa23-4402-9751-80dd38c89927 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:05.969278 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 434/868] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:14:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:05.974929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9699ec-fa23-4402-9751-80dd38c89927 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:05.983537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 435/869] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:05 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 19:14:06.006018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb588f94-33be-4e6e-a5c8-0d7098966add None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:06.042993 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:06.108955 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9699ec-fa23-4402-9751-80dd38c89927 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:06.125912 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:06.141466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb588f94-33be-4e6e-a5c8-0d7098966add None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:06.144874 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fa9da-8ac4-4f93-8a15-0eb1ce283ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:06.145500 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fa9da-8ac4-4f93-8a15-0eb1ce283ffa None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:06.145781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fa9da-8ac4-4f93-8a15-0eb1ce283ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:06.146046 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5fa9da-8ac4-4f93-8a15-0eb1ce283ffa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:06.147559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fa9da-8ac4-4f93-8a15-0eb1ce283ffa None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=db51971158a946c98a8ed716ecc5d72b) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:06.147559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fa9da-8ac4-4f93-8a15-0eb1ce283ffa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:06.148570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5fa9da-8ac4-4f93-8a15-0eb1ce283ffa None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:06.155434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 435/870] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:14:05 2022] GET /identity/v3/role_assignments?user.id=db51971158a946c98a8ed716ecc5d72b&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=4da42f08d107448fb877b42b61bc74ea => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:06.194584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf834a6b-eef2-4f39-8e21-674266c94b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:06.195260 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf834a6b-eef2-4f39-8e21-674266c94b7d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:06.195543 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf834a6b-eef2-4f39-8e21-674266c94b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:06.195819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf834a6b-eef2-4f39-8e21-674266c94b7d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:06.196513 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf834a6b-eef2-4f39-8e21-674266c94b7d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:06.197943 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf834a6b-eef2-4f39-8e21-674266c94b7d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:06.208161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-bf834a6b-eef2-4f39-8e21-674266c94b7d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 19:14:06.208161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:06.208161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:06.208161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:06.208161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:06.208161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:06.208161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:06.208161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:06.208161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 19:14:06.208161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:06.208161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:06.208161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:06.208161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:06.208161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:06.208161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:06.208161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:06.209241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:06.209241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:06.209241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:06.209241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:06.209241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:06.209241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:06.209241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:06.209241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:06.209241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:06.209241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:06.209241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:06.209241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:06.209241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:06.209241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:06.209241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:06.209241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:06.209241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:06.210489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:06.210489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:06.210489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:06.210489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:06.210489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:06.210489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:06.210489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:06.210489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:06.210489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:06.210489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:06.210489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:06.210489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:06.210489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 19:14:06.210489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:06.210489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 436/871] 10.222.0.17 () {56 vars in 1132 bytes} [Sun Aug 7 19:14:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:06.218610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fa9da-8ac4-4f93-8a15-0eb1ce283ffa None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:06.335193 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5fa9da-8ac4-4f93-8a15-0eb1ce283ffa None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:06.345311 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:06.371709 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8948745d-eb37-4865-bc51-6e20adbb5bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:06.372372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948745d-eb37-4865-bc51-6e20adbb5bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:06.372651 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948745d-eb37-4865-bc51-6e20adbb5bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:06.372922 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8948745d-eb37-4865-bc51-6e20adbb5bc1 None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:06.373566 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948745d-eb37-4865-bc51-6e20adbb5bc1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:06.373868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948745d-eb37-4865-bc51-6e20adbb5bc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:06.375261 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8948745d-eb37-4865-bc51-6e20adbb5bc1 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:06.380932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 436/872] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Aug 7 19:14:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:06.389129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf834a6b-eef2-4f39-8e21-674266c94b7d None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:06.504976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf834a6b-eef2-4f39-8e21-674266c94b7d None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:06.542671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3786bc2c-789d-4160-b087-9dfdf1ce8bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:06.543322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786bc2c-789d-4160-b087-9dfdf1ce8bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:06.543604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786bc2c-789d-4160-b087-9dfdf1ce8bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:06.543877 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3786bc2c-789d-4160-b087-9dfdf1ce8bb8 None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:06.544584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786bc2c-789d-4160-b087-9dfdf1ce8bb8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:06.544895 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786bc2c-789d-4160-b087-9dfdf1ce8bb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:06.546332 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3786bc2c-789d-4160-b087-9dfdf1ce8bb8 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:06.552536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 437/873] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Aug 7 19:14:06 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:07.449466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea870d92-2390-4557-8cd0-bad48330fb2b None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:07.450020 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea870d92-2390-4557-8cd0-bad48330fb2b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:07.450020 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea870d92-2390-4557-8cd0-bad48330fb2b None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:07.451594 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 437/874] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:07.457254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bd40e-3e1f-404a-b975-613fb7d351ea None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:07.457692 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bd40e-3e1f-404a-b975-613fb7d351ea None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:07.457873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bd40e-3e1f-404a-b975-613fb7d351ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:07.476636 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-d02bd40e-3e1f-404a-b975-613fb7d351ea None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:07.477838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d02bd40e-3e1f-404a-b975-613fb7d351ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:07.489793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:07.548485 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 438/875] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:07 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:07.571616 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea870d92-2390-4557-8cd0-bad48330fb2b None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:07.683822 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea870d92-2390-4557-8cd0-bad48330fb2b None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:07.720734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9706f-27d7-4c9e-8d20-07a4929fc133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:07.721293 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9706f-27d7-4c9e-8d20-07a4929fc133 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:07.721624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9706f-27d7-4c9e-8d20-07a4929fc133 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:07.721897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9706f-27d7-4c9e-8d20-07a4929fc133 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:07.732747 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9706f-27d7-4c9e-8d20-07a4929fc133 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:14:07.737482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-39e9706f-27d7-4c9e-8d20-07a4929fc133 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 19:14:07.737482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:07.737482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:07.737482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:07.737482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:07.737482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:07.737482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:07.737482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:07.737482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:07.737482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:07.737482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:07.737482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:07.737482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:14:07.737482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:14:07.737482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:14:07.737482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:14:07.738415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:14:07.738415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:14:07.738415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:14:07.738415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:14:07.738415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:07.738415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:07.738415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:07.738415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:07.738415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:07.738415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:07.738415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:07.738415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:07.738415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:07.738415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:07.738415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:07.738415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:07.739859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:07.739859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:07.739859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:07.739859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:07.739859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:14:07.739859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:14:07.739859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:14:07.739859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:14:07.739859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:14:07.739859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:14:07.739859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 19:14:07.739859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:07.739859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 438/876] 10.222.0.17 () {56 vars in 1120 bytes} [Sun Aug 7 19:14:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:07.744950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02bd40e-3e1f-404a-b975-613fb7d351ea None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:07.851714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02bd40e-3e1f-404a-b975-613fb7d351ea None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:07.886802 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f95279-7e31-4612-8163-d14c976b034c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:07.887434 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f95279-7e31-4612-8163-d14c976b034c None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:07.887717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f95279-7e31-4612-8163-d14c976b034c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:07.888013 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f95279-7e31-4612-8163-d14c976b034c None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:07.888703 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f95279-7e31-4612-8163-d14c976b034c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:07.889151 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f95279-7e31-4612-8163-d14c976b034c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:07.890540 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f95279-7e31-4612-8163-d14c976b034c None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:07.899447 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 439/877] 10.222.0.17 () {56 vars in 1115 bytes} [Sun Aug 7 19:14:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:07.909157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9706f-27d7-4c9e-8d20-07a4929fc133 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:07.942904 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:08.020345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9706f-27d7-4c9e-8d20-07a4929fc133 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:08.056018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e11c0-f854-4245-a847-f31cd1894dee None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:08.056374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e11c0-f854-4245-a847-f31cd1894dee None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:08.056575 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e11c0-f854-4245-a847-f31cd1894dee None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:08.058497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 440/878] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:14:08.060922 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5cddd-b6fd-4ad8-a71e-0f700a3d5302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:08.061571 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5cddd-b6fd-4ad8-a71e-0f700a3d5302 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:08.061847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5cddd-b6fd-4ad8-a71e-0f700a3d5302 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:08.062164 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5cddd-b6fd-4ad8-a71e-0f700a3d5302 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:08.064924 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcaabb-887c-4630-80ef-54fc7eeac939 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:08.065461 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcaabb-887c-4630-80ef-54fc7eeac939 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:08.065957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcaabb-887c-4630-80ef-54fc7eeac939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:08.073367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5cddd-b6fd-4ad8-a71e-0f700a3d5302 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:08.076138 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5cddd-b6fd-4ad8-a71e-0f700a3d5302 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:08.088273 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-4ddcaabb-887c-4630-80ef-54fc7eeac939 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:08.089536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4ddcaabb-887c-4630-80ef-54fc7eeac939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:08.091638 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:08.092372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-1ef5cddd-b6fd-4ad8-a71e-0f700a3d5302 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:08.092372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:08.092372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:08.092372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:08.092372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:08.092372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:08.092372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:08.092372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:08.092372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:08.092372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:08.092372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:08.092372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:08.092372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:08.092372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:08.092372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:08.092372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:08.093322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:08.093322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:08.093322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:08.093322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:08.093322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:08.093322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:08.093322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:08.093322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:08.093322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:08.093322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:08.093322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:08.093322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:08.093322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:08.093322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:08.093322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:08.093322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:08.093322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:08.094218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:08.094218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:08.094218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:08.101217 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 439/879] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:08.102545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5cddd-b6fd-4ad8-a71e-0f700a3d5302 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:08.180087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 441/880] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:08 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 19:14:08.198432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcaabb-887c-4630-80ef-54fc7eeac939 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:08.203315 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:08.221217 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5cddd-b6fd-4ad8-a71e-0f700a3d5302 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:08.257974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57afe639-7131-4fce-84c0-bc46c036908d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:08.258891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afe639-7131-4fce-84c0-bc46c036908d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:08.259172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afe639-7131-4fce-84c0-bc46c036908d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:08.259442 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57afe639-7131-4fce-84c0-bc46c036908d None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:08.260130 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afe639-7131-4fce-84c0-bc46c036908d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:08.260434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afe639-7131-4fce-84c0-bc46c036908d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:08.261972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57afe639-7131-4fce-84c0-bc46c036908d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:08.276151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 440/881] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:08.281330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54337d-9a85-4cf0-b01c-6bc4bd502074 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:08.281768 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54337d-9a85-4cf0-b01c-6bc4bd502074 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:08.282165 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54337d-9a85-4cf0-b01c-6bc4bd502074 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:08.283795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 441/882] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:08.288439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54337d-9a85-4cf0-b01c-6bc4bd502074 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:08.308657 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcaabb-887c-4630-80ef-54fc7eeac939 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:08.345870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb0376-f7de-4085-b2ce-df73d0417e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:08.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb0376-f7de-4085-b2ce-df73d0417e5b None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:08.346985 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb0376-f7de-4085-b2ce-df73d0417e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:08.347258 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb0376-f7de-4085-b2ce-df73d0417e5b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:08.347888 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb0376-f7de-4085-b2ce-df73d0417e5b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:08.349304 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb0376-f7de-4085-b2ce-df73d0417e5b None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:08.358382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-d1cb0376-f7de-4085-b2ce-df73d0417e5b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 19:14:08.358382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:08.358382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:08.358382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:08.358382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:08.358382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:08.358382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:08.358382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:08.358382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 19:14:08.358382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:08.358382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:08.358382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:08.358382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:08.358382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:08.358382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:08.358382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:08.358382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:08.359409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:08.359409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:08.359409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:08.359409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:08.359409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:08.359409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:08.359409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:08.359409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:08.359409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:08.359409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:08.359409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:08.359409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:08.359409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:08.359409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:08.359409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:08.359409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:08.360290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:08.360290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:08.360290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:08.360290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:08.360290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:08.360290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:08.360290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:08.360290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:08.360290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:08.360290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:08.360290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:08.360290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:08.360290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 19:14:08.360290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:08.360290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 442/883] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:14:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:08.361086 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ffc39-ef9e-4a1c-9a41-f0d0467c8d71 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:08.361086 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ffc39-ef9e-4a1c-9a41-f0d0467c8d71 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:08.361780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ffc39-ef9e-4a1c-9a41-f0d0467c8d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:08.367854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:08.382653 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-3c0ffc39-ef9e-4a1c-9a41-f0d0467c8d71 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:08.384143 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-3c0ffc39-ef9e-4a1c-9a41-f0d0467c8d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:08.403911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54337d-9a85-4cf0-b01c-6bc4bd502074 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:08.442129 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec496b-d87c-418c-9576-7a513229f665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:08.442794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec496b-d87c-418c-9576-7a513229f665 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:08.443088 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec496b-d87c-418c-9576-7a513229f665 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:08.443342 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec496b-d87c-418c-9576-7a513229f665 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:08.452091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec496b-d87c-418c-9576-7a513229f665 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:08.453592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ec496b-d87c-418c-9576-7a513229f665 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:08.457406 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 443/884] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:08 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:08.457869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ffc39-ef9e-4a1c-9a41-f0d0467c8d71 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:08.485219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-78ec496b-d87c-418c-9576-7a513229f665 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 19:14:08.485219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:08.485219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:08.485219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:08.485219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:08.485219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:08.485219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:08.485219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:08.485219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:08.485219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:08.485219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:08.485219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:08.485219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:14:08.485219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:14:08.485219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:14:08.485219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:08.486350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:08.487299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:08.487299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:08.487299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:14:08.487299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:14:08.487299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:14:08.487299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:14:08.487299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:14:08.487299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:14:08.487299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 19:14:08.487299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:08.487299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 442/885] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Aug 7 19:14:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:08.487919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec496b-d87c-418c-9576-7a513229f665 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:08.496840 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:08.585449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ffc39-ef9e-4a1c-9a41-f0d0467c8d71 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:08.613744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec496b-d87c-418c-9576-7a513229f665 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:08.616371 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:08.621840 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c7409-d0dd-4bec-a799-0f97a4d6400a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:08.622469 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c7409-d0dd-4bec-a799-0f97a4d6400a None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:08.622742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c7409-d0dd-4bec-a799-0f97a4d6400a None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:08.623001 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c7409-d0dd-4bec-a799-0f97a4d6400a None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:08.623628 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c7409-d0dd-4bec-a799-0f97a4d6400a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:08.623926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c7409-d0dd-4bec-a799-0f97a4d6400a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:08.625320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c7409-d0dd-4bec-a799-0f97a4d6400a None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:08.630486 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 444/886] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:14:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:08.631267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c7409-d0dd-4bec-a799-0f97a4d6400a None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:08.653196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4f3c2-f917-45ad-9f59-eed8555d0b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:08.654254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4f3c2-f917-45ad-9f59-eed8555d0b06 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:08.654523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4f3c2-f917-45ad-9f59-eed8555d0b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:08.654791 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4f3c2-f917-45ad-9f59-eed8555d0b06 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:08.655389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f4f3c2-f917-45ad-9f59-eed8555d0b06 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:08.656795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f4f3c2-f917-45ad-9f59-eed8555d0b06 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:08.665415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-20f4f3c2-f917-45ad-9f59-eed8555d0b06 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 19:14:08.665415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:08.665415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:08.665415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:08.665415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:08.665415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:08.665415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:08.665415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:08.665415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 19:14:08.665415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:08.665415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:08.665415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:08.665415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:08.665415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:08.665415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:08.665415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:08.665415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:08.666405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:08.666405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:08.666405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:08.666405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:08.666405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:08.666405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:08.666405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:08.666405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:08.666405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:08.666405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:08.666405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:08.666405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:08.666405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:08.666405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:08.666405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:08.666405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:08.667272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:08.667272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:08.667272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:08.667272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:08.667272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:08.667272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:08.667272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:08.667272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:08.667272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:08.667272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:08.667272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:08.667272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:08.667272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 19:14:08.667272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:08.667272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 443/887] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Aug 7 19:14:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:08.668150 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a0ce4-ccec-4279-8e79-fa9c209ec318 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:08.668150 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a0ce4-ccec-4279-8e79-fa9c209ec318 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:08.668809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a0ce4-ccec-4279-8e79-fa9c209ec318 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:08.669697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 444/888] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:08.671748 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a0ce4-ccec-4279-8e79-fa9c209ec318 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:08.739171 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c7409-d0dd-4bec-a799-0f97a4d6400a None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:08.740095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:08.774003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e47446-b76d-4cc3-b905-665d87deb785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:08.774631 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e47446-b76d-4cc3-b905-665d87deb785 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:08.774911 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e47446-b76d-4cc3-b905-665d87deb785 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:08.775187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e47446-b76d-4cc3-b905-665d87deb785 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:08.775924 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e47446-b76d-4cc3-b905-665d87deb785 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:08.776217 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a0ce4-ccec-4279-8e79-fa9c209ec318 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:08.776356 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e47446-b76d-4cc3-b905-665d87deb785 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:08.777659 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e47446-b76d-4cc3-b905-665d87deb785 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:08.796314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 445/889] 10.222.0.17 () {56 vars in 1149 bytes} [Sun Aug 7 19:14:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:08.797172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e47446-b76d-4cc3-b905-665d87deb785 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:08.814495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f794137-e591-4c85-b294-0821bd3f9ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:08.815119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f794137-e591-4c85-b294-0821bd3f9ddc None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:08.815393 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f794137-e591-4c85-b294-0821bd3f9ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:08.815660 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f794137-e591-4c85-b294-0821bd3f9ddc None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:08.816341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f794137-e591-4c85-b294-0821bd3f9ddc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:08.816642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f794137-e591-4c85-b294-0821bd3f9ddc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:08.818034 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f794137-e591-4c85-b294-0821bd3f9ddc None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:08.823074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 445/890] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:14:08 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:08.824563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085146b-8ed2-4027-830f-dc7939fbe550 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:08.824846 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085146b-8ed2-4027-830f-dc7939fbe550 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:08.825114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085146b-8ed2-4027-830f-dc7939fbe550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:08.843699 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-0085146b-8ed2-4027-830f-dc7939fbe550 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:08.844872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0085146b-8ed2-4027-830f-dc7939fbe550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:08.868720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:08.916826 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 446/891] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:08 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:08.917943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085146b-8ed2-4027-830f-dc7939fbe550 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:08.922640 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e47446-b76d-4cc3-b905-665d87deb785 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:08.957382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300ca2b-4275-4404-972a-f746e9a0061c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:08.958005 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300ca2b-4275-4404-972a-f746e9a0061c None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:08.958287 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300ca2b-4275-4404-972a-f746e9a0061c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:08.958560 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300ca2b-4275-4404-972a-f746e9a0061c None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:08.959199 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300ca2b-4275-4404-972a-f746e9a0061c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:08.959502 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300ca2b-4275-4404-972a-f746e9a0061c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:08.960907 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5300ca2b-4275-4404-972a-f746e9a0061c None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:08.965491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 446/892] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Aug 7 19:14:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:08.966285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300ca2b-4275-4404-972a-f746e9a0061c None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:08.983002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:09.023427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085146b-8ed2-4027-830f-dc7939fbe550 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:09.060634 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdecda-c08a-4632-b53f-724ce1cdf7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:09.061255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdecda-c08a-4632-b53f-724ce1cdf7cc None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:09.061508 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdecda-c08a-4632-b53f-724ce1cdf7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:09.061773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdecda-c08a-4632-b53f-724ce1cdf7cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:09.071487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5300ca2b-4275-4404-972a-f746e9a0061c None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:09.088909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:09.106577 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d613e-8141-42d1-a94d-a7b6f44e2ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:09.107112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d613e-8141-42d1-a94d-a7b6f44e2ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:09.107388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d613e-8141-42d1-a94d-a7b6f44e2ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:09.107653 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d613e-8141-42d1-a94d-a7b6f44e2ee9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:09.108444 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d613e-8141-42d1-a94d-a7b6f44e2ee9 None admin] RBAC: Adding query filter params (interface=public, service_id=e3ef6f5dc6454063a436bda3eaf6fffd, region_id=RegionOne) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:09.108744 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d613e-8141-42d1-a94d-a7b6f44e2ee9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:09.110113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d613e-8141-42d1-a94d-a7b6f44e2ee9 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:09.115273 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 447/893] 10.222.0.17 () {56 vars in 1251 bytes} [Sun Aug 7 19:14:08 2022] GET /identity/v3/endpoints?service_id=e3ef6f5dc6454063a436bda3eaf6fffd&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:09.116089 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d613e-8141-42d1-a94d-a7b6f44e2ee9 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:09.130523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbdecda-c08a-4632-b53f-724ce1cdf7cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:09.132010 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbdecda-c08a-4632-b53f-724ce1cdf7cc None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:09.204445 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:09.222248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d613e-8141-42d1-a94d-a7b6f44e2ee9 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:09.258085 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f8006-03d4-4c4f-bc5d-0b2c52bd05b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:09.259142 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f8006-03d4-4c4f-bc5d-0b2c52bd05b8 None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:09.259547 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f8006-03d4-4c4f-bc5d-0b2c52bd05b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:09.259825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f8006-03d4-4c4f-bc5d-0b2c52bd05b8 None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:09.260518 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f8006-03d4-4c4f-bc5d-0b2c52bd05b8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:09.261925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f8006-03d4-4c4f-bc5d-0b2c52bd05b8 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:09.265611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 447/894] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:14:08 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:14:09.266494 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdecda-c08a-4632-b53f-724ce1cdf7cc None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:09.293254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 448/895] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Aug 7 19:14:09 2022] POST /identity/v3/services => generated 269 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:14:09.293254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f8006-03d4-4c4f-bc5d-0b2c52bd05b8 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:09.337237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:09.404340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdecda-c08a-4632-b53f-724ce1cdf7cc None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:09.409651 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f8006-03d4-4c4f-bc5d-0b2c52bd05b8 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:09.445770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b91a2b-2328-4fa3-b658-771e6434e9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:09.446397 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b91a2b-2328-4fa3-b658-771e6434e9ac None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:09.446731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b91a2b-2328-4fa3-b658-771e6434e9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:09.447003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b91a2b-2328-4fa3-b658-771e6434e9ac None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:09.447672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b91a2b-2328-4fa3-b658-771e6434e9ac None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:09.447975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b91a2b-2328-4fa3-b658-771e6434e9ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:09.449436 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b91a2b-2328-4fa3-b658-771e6434e9ac None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:09.450277 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef79b4f-fbc9-4490-9b15-49ec2559be66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:09.450904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef79b4f-fbc9-4490-9b15-49ec2559be66 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:09.451187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef79b4f-fbc9-4490-9b15-49ec2559be66 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:09.451461 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef79b4f-fbc9-4490-9b15-49ec2559be66 None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:09.452106 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef79b4f-fbc9-4490-9b15-49ec2559be66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:09.453476 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef79b4f-fbc9-4490-9b15-49ec2559be66 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:09.455623 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 448/896] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Aug 7 19:14:09 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:09.456494 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b91a2b-2328-4fa3-b658-771e6434e9ac None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:09.460351 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 449/897] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Aug 7 19:14:09 2022] GET /identity/v3/services => generated 1268 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:14:09.493352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:09.594227 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b91a2b-2328-4fa3-b658-771e6434e9ac None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:09.640923 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0d322-9395-48db-94a2-3b92b1cce82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:09.641589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0d322-9395-48db-94a2-3b92b1cce82d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:09.641863 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0d322-9395-48db-94a2-3b92b1cce82d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:09.642128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0d322-9395-48db-94a2-3b92b1cce82d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:09.655112 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0d322-9395-48db-94a2-3b92b1cce82d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:09.656537 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e0d322-9395-48db-94a2-3b92b1cce82d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:09.672787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-d8e0d322-9395-48db-94a2-3b92b1cce82d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:09.672787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:09.672787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:09.672787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:09.672787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:09.672787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:09.672787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:09.672787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:09.672787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:09.672787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:09.672787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:09.672787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:09.672787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:09.672787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:09.672787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:09.672787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:09.675312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:09.675312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:09.675312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:09.675312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:09.675312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:09.675312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:09.675312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:09.675312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:09.675312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:09.675312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:09.675312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:09.675312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:09.675312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:09.675312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:09.675312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:09.675312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:09.675312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:09.676730 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:09.676730 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:09.676730 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:09.676730 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 449/898] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:09.680018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef79b4f-fbc9-4490-9b15-49ec2559be66 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:09.789973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef79b4f-fbc9-4490-9b15-49ec2559be66 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:09.791639 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:09.831755 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5dfae-a6f7-4a81-b8b2-1199fd0408aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:09.832667 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5dfae-a6f7-4a81-b8b2-1199fd0408aa None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:09.833097 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5dfae-a6f7-4a81-b8b2-1199fd0408aa None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:09.833536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5dfae-a6f7-4a81-b8b2-1199fd0408aa None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:09.834430 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5dfae-a6f7-4a81-b8b2-1199fd0408aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:09.834908 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5dfae-a6f7-4a81-b8b2-1199fd0408aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:09.836477 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5dfae-a6f7-4a81-b8b2-1199fd0408aa None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:09.850280 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 450/899] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:09.856017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0d322-9395-48db-94a2-3b92b1cce82d None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:09.956275 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:09.992801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0d322-9395-48db-94a2-3b92b1cce82d None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:10.039901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59668bc-7331-4777-a855-a812403f28f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:10.040721 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59668bc-7331-4777-a855-a812403f28f3 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:10.041153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59668bc-7331-4777-a855-a812403f28f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:10.041516 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59668bc-7331-4777-a855-a812403f28f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:10.048886 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59668bc-7331-4777-a855-a812403f28f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:10.050461 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59668bc-7331-4777-a855-a812403f28f3 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:10.062680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-a59668bc-7331-4777-a855-a812403f28f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:10.062680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:10.062680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:10.062680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:10.062680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:10.062680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:10.062680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:10.062680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:10.062680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:10.062680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:10.062680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:10.062680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:10.062680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:14:10.062680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:14:10.062680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:14:10.063835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:14:10.063835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:10.063835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:10.063835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:10.063835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:10.063835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:10.063835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:10.063835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:10.063835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:10.063835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:10.063835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:10.063835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:10.063835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:10.063835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:10.063835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:10.063835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:10.063835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:10.064829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:14:10.064829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:14:10.064829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:14:10.064829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:14:10.064829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:14:10.064829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:14:10.064829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:10.064829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:10.065573 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 450/900] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:14:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:10.071159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5dfae-a6f7-4a81-b8b2-1199fd0408aa None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:10.179863 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5dfae-a6f7-4a81-b8b2-1199fd0408aa None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:10.221122 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dcd10-3bf4-476e-96ba-e94c2ce11198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:10.221725 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dcd10-3bf4-476e-96ba-e94c2ce11198 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:10.222009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dcd10-3bf4-476e-96ba-e94c2ce11198 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:10.222284 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dcd10-3bf4-476e-96ba-e94c2ce11198 None admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:10.223076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dcd10-3bf4-476e-96ba-e94c2ce11198 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:10.223388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dcd10-3bf4-476e-96ba-e94c2ce11198 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:10.225025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899dcd10-3bf4-476e-96ba-e94c2ce11198 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:10.238536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 451/901] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:14:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:10.244144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59668bc-7331-4777-a855-a812403f28f3 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:10.349801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59668bc-7331-4777-a855-a812403f28f3 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:10.385074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d7f11-69d8-4f90-b9b9-1b5d08acff9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:10.386164 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d7f11-69d8-4f90-b9b9-1b5d08acff9c None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:10.386666 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d7f11-69d8-4f90-b9b9-1b5d08acff9c None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:10.387152 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d7f11-69d8-4f90-b9b9-1b5d08acff9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:10.388130 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d7f11-69d8-4f90-b9b9-1b5d08acff9c None admin] RBAC: Adding query filter params (role.id=c5946f310572439687baeed49acd4336, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=61ff29258a214817b8fc15b51fe8b3e6) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:10.388638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d7f11-69d8-4f90-b9b9-1b5d08acff9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:10.390294 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833d7f11-69d8-4f90-b9b9-1b5d08acff9c None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:10.396418 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 451/902] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:14:10 2022] GET /identity/v3/role_assignments?user.id=61ff29258a214817b8fc15b51fe8b3e6&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=c5946f310572439687baeed49acd4336 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:11.484495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a32b31-5d7b-4616-a215-e6c93e7876e7 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:11.485545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a32b31-5d7b-4616-a215-e6c93e7876e7 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:11.486042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a32b31-5d7b-4616-a215-e6c93e7876e7 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:11.487909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 452/903] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:11.493591 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177e89c-63e4-4793-a2e6-278f79a90bd2 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:11.494458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177e89c-63e4-4793-a2e6-278f79a90bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:11.494955 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177e89c-63e4-4793-a2e6-278f79a90bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:11.501721 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:11.516533 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-d177e89c-63e4-4793-a2e6-278f79a90bd2 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:11.518188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-d177e89c-63e4-4793-a2e6-278f79a90bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:11.593692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 452/904] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:11.606246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7fb07-1afa-44c8-9f19-e8675559b8e5 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:11.606585 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7fb07-1afa-44c8-9f19-e8675559b8e5 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:11.606810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7fb07-1afa-44c8-9f19-e8675559b8e5 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:11.611489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 453/905] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:11.611624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22617c57-a258-4088-baf2-98dc8f28a497 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:11.611624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22617c57-a258-4088-baf2-98dc8f28a497 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:11.611624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22617c57-a258-4088-baf2-98dc8f28a497 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:11.612175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 453/906] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:11.613624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df882b2-e893-4c83-b910-9c1382ff1a06 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:11.613911 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df882b2-e893-4c83-b910-9c1382ff1a06 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:11.614186 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df882b2-e893-4c83-b910-9c1382ff1a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:11.617410 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0af05-4055-4ae4-b480-bc2e0d604aaa None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:11.618023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0af05-4055-4ae4-b480-bc2e0d604aaa None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:11.618512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0af05-4055-4ae4-b480-bc2e0d604aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:11.634103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-9df882b2-e893-4c83-b910-9c1382ff1a06 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:11.635297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9df882b2-e893-4c83-b910-9c1382ff1a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:11.637624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-dcd0af05-4055-4ae4-b480-bc2e0d604aaa None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:11.638993 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-dcd0af05-4055-4ae4-b480-bc2e0d604aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:11.653368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:11.717649 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 454/907] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:11.718808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df882b2-e893-4c83-b910-9c1382ff1a06 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:11.736125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 454/908] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:11.747461 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0af05-4055-4ae4-b480-bc2e0d604aaa None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:11.834851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:11.867224 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df882b2-e893-4c83-b910-9c1382ff1a06 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:11.890109 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0af05-4055-4ae4-b480-bc2e0d604aaa None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:11.903337 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d3366-06e5-4642-944c-ba686b182101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:11.904124 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d3366-06e5-4642-944c-ba686b182101 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:11.904473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d3366-06e5-4642-944c-ba686b182101 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:11.904811 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d3366-06e5-4642-944c-ba686b182101 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:11.905528 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d3366-06e5-4642-944c-ba686b182101 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:11.907005 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d3366-06e5-4642-944c-ba686b182101 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:11.919014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-8d0d3366-06e5-4642-944c-ba686b182101 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 19:14:11.919014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:11.919014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:11.919014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:11.919014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:11.919014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:11.919014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:11.919014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:11.919014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 19:14:11.919014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:11.919014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:11.919014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:11.919014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:11.919014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:11.919014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:11.919014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:11.920022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:11.920022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:11.920022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:11.920022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:11.920022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:11.920022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:11.920022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:11.920022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:11.920022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:11.920022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:11.920022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:11.920022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:11.920022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:11.920022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:11.920022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:11.920022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:11.920022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:11.920920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:11.920920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:11.920920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:11.920920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:11.920920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:11.920920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:11.920920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:11.920920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:11.920920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:11.920920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:11.920920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:11.920920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:11.920920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 19:14:11.920920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:11.920920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 455/909] 10.222.0.17 () {56 vars in 1132 bytes} [Sun Aug 7 19:14:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:11.921710 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d3366-06e5-4642-944c-ba686b182101 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:11.940158 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d1a50-909f-483a-b0b1-790dd41b2500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:11.940803 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d1a50-909f-483a-b0b1-790dd41b2500 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:11.941084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d1a50-909f-483a-b0b1-790dd41b2500 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:11.941358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d1a50-909f-483a-b0b1-790dd41b2500 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:11.941951 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d1a50-909f-483a-b0b1-790dd41b2500 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:11.943397 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d1a50-909f-483a-b0b1-790dd41b2500 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:11.951093 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:11.952655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-2d3d1a50-909f-483a-b0b1-790dd41b2500 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 19:14:11.952655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:11.952655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:11.952655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:11.952655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:11.952655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:11.952655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:11.952655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:11.952655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 19:14:11.952655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:11.952655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:11.952655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:11.952655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:11.952655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:11.952655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:11.952655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:11.952655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:11.953700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:11.953700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:11.953700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:11.953700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:11.953700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:11.953700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:11.953700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:11.953700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:11.953700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:11.953700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:11.953700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:11.953700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:11.953700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:11.953700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:11.953700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:11.953700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:11.954534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:11.954534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:11.954534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:11.954534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:11.954534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:11.954534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:11.954534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:11.954534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:11.954534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:11.954534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:11.954534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:11.954534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:11.954534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 19:14:11.954534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:11.959468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 455/910] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:14:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:11.960495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d1a50-909f-483a-b0b1-790dd41b2500 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:12.065133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d3366-06e5-4642-944c-ba686b182101 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:12.071490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d1a50-909f-483a-b0b1-790dd41b2500 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:12.098986 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:12.106307 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e49c4-7d52-406a-9c73-bc8122fbfa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:12.106974 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e49c4-7d52-406a-9c73-bc8122fbfa1d None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:12.107201 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e49c4-7d52-406a-9c73-bc8122fbfa1d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:12.107469 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e49c4-7d52-406a-9c73-bc8122fbfa1d None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:12.108183 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e49c4-7d52-406a-9c73-bc8122fbfa1d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:12.109570 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e49c4-7d52-406a-9c73-bc8122fbfa1d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:12.112391 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b176219-c082-44f3-9943-66085576a861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:12.113656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b176219-c082-44f3-9943-66085576a861 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:12.114237 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b176219-c082-44f3-9943-66085576a861 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:12.115197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b176219-c082-44f3-9943-66085576a861 None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:12.116108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b176219-c082-44f3-9943-66085576a861 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:12.116813 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b176219-c082-44f3-9943-66085576a861 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:12.118981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b176219-c082-44f3-9943-66085576a861 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:12.127340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 456/911] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Aug 7 19:14:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:12.129402 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b176219-c082-44f3-9943-66085576a861 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:12.131953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 456/912] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Aug 7 19:14:11 2022] POST /identity/v3/services => generated 241 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:14:12.133002 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e49c4-7d52-406a-9c73-bc8122fbfa1d None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:12.261865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:12.281132 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b176219-c082-44f3-9943-66085576a861 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:12.305362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e49c4-7d52-406a-9c73-bc8122fbfa1d None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:12.329575 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ad0c7-37cf-448b-8304-027582943ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:12.330535 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ad0c7-37cf-448b-8304-027582943ebe None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:12.330999 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ad0c7-37cf-448b-8304-027582943ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:12.331454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ad0c7-37cf-448b-8304-027582943ebe None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:12.332346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ad0c7-37cf-448b-8304-027582943ebe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:12.332851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ad0c7-37cf-448b-8304-027582943ebe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:12.334471 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ad0c7-37cf-448b-8304-027582943ebe None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:12.341764 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 457/913] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:14:12 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:12.347820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ad0c7-37cf-448b-8304-027582943ebe None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:12.356739 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de27c3-7c1a-4676-801e-9d0cde1bb2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:12.357369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de27c3-7c1a-4676-801e-9d0cde1bb2ea None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:12.357649 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de27c3-7c1a-4676-801e-9d0cde1bb2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:12.357922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de27c3-7c1a-4676-801e-9d0cde1bb2ea None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:12.359091 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de27c3-7c1a-4676-801e-9d0cde1bb2ea None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:12.359404 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de27c3-7c1a-4676-801e-9d0cde1bb2ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:12.361412 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de27c3-7c1a-4676-801e-9d0cde1bb2ea None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:12.369070 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 457/914] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Aug 7 19:14:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:12.375426 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de27c3-7c1a-4676-801e-9d0cde1bb2ea None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:12.411874 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:12.467533 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ad0c7-37cf-448b-8304-027582943ebe None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:12.490554 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de27c3-7c1a-4676-801e-9d0cde1bb2ea None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:12.504097 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644e1f2-f2b6-4e7d-8a0b-97ddf24c4d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:12.504748 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644e1f2-f2b6-4e7d-8a0b-97ddf24c4d53 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:12.504988 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644e1f2-f2b6-4e7d-8a0b-97ddf24c4d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:12.505255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644e1f2-f2b6-4e7d-8a0b-97ddf24c4d53 None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:12.505889 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644e1f2-f2b6-4e7d-8a0b-97ddf24c4d53 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:12.506183 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644e1f2-f2b6-4e7d-8a0b-97ddf24c4d53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:12.507574 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644e1f2-f2b6-4e7d-8a0b-97ddf24c4d53 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:12.517514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 458/915] 10.222.0.17 () {56 vars in 1114 bytes} [Sun Aug 7 19:14:12 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:12.517514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b400df-a2e2-4e9e-8708-2cad727d48de None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:12.517514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b400df-a2e2-4e9e-8708-2cad727d48de None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:12.517514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b400df-a2e2-4e9e-8708-2cad727d48de None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:12.517514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 459/916] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:12.521007 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b400df-a2e2-4e9e-8708-2cad727d48de None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:12.533649 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f64bd58-6ebc-465b-956f-bb6c903715c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:12.534279 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64bd58-6ebc-465b-956f-bb6c903715c4 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:12.534563 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64bd58-6ebc-465b-956f-bb6c903715c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:12.534836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f64bd58-6ebc-465b-956f-bb6c903715c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:12.535550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64bd58-6ebc-465b-956f-bb6c903715c4 None admin] RBAC: Adding query filter params (interface=public, service_id=8dfe39595b2a44ddb0ebc5e4f0dbdd2e, region_id=RegionOne) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:12.535862 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64bd58-6ebc-465b-956f-bb6c903715c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:12.537294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f64bd58-6ebc-465b-956f-bb6c903715c4 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:12.540362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:12.548777 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 458/917] 10.222.0.17 () {56 vars in 1251 bytes} [Sun Aug 7 19:14:12 2022] GET /identity/v3/endpoints?service_id=8dfe39595b2a44ddb0ebc5e4f0dbdd2e&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:12.550254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a08a4a-9b4c-4a13-8dbb-287d33ccaeec None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:12.550469 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a08a4a-9b4c-4a13-8dbb-287d33ccaeec None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:12.550745 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a08a4a-9b4c-4a13-8dbb-287d33ccaeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:12.569757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-49a08a4a-9b4c-4a13-8dbb-287d33ccaeec None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:12.570979 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-49a08a4a-9b4c-4a13-8dbb-287d33ccaeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:12.630859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b400df-a2e2-4e9e-8708-2cad727d48de None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:12.650404 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:12.656300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 459/918] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:12.657151 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a08a4a-9b4c-4a13-8dbb-287d33ccaeec None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:12.676779 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579e4a6-c7e3-4647-b4c6-85205f2bd4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:12.677411 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579e4a6-c7e3-4647-b4c6-85205f2bd4e6 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:12.677688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579e4a6-c7e3-4647-b4c6-85205f2bd4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:12.677973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579e4a6-c7e3-4647-b4c6-85205f2bd4e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:12.678618 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579e4a6-c7e3-4647-b4c6-85205f2bd4e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:12.680029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579e4a6-c7e3-4647-b4c6-85205f2bd4e6 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:12.701941 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 460/919] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:14:12 2022] POST /identity/v3/endpoints => generated 346 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:14:12.702606 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579e4a6-c7e3-4647-b4c6-85205f2bd4e6 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:12.800463 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a08a4a-9b4c-4a13-8dbb-287d33ccaeec None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:12.846060 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:12.862450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75babe-ced7-4112-866d-36835aa150a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:12.863876 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75babe-ced7-4112-866d-36835aa150a8 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:12.864322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75babe-ced7-4112-866d-36835aa150a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:12.864693 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75babe-ced7-4112-866d-36835aa150a8 None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:12.866114 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75babe-ced7-4112-866d-36835aa150a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:12.867947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75babe-ced7-4112-866d-36835aa150a8 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:12.869344 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579e4a6-c7e3-4647-b4c6-85205f2bd4e6 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:12.876293 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 460/920] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Aug 7 19:14:12 2022] GET /identity/v3/services => generated 1497 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:14:12.920814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa17f9-dd46-4977-8c27-2d1633df59f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:12.921827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa17f9-dd46-4977-8c27-2d1633df59f0 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:12.922369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa17f9-dd46-4977-8c27-2d1633df59f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:12.922846 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa17f9-dd46-4977-8c27-2d1633df59f0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:12.937265 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa17f9-dd46-4977-8c27-2d1633df59f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:14:12.942859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-87fa17f9-dd46-4977-8c27-2d1633df59f0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 19:14:12.942859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:12.942859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:12.942859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:12.942859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:12.942859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:12.942859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:12.942859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:12.942859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:12.942859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:12.942859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:12.942859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:12.942859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:14:12.942859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:14:12.942859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:14:12.942859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:14:12.944238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:14:12.944238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:14:12.944238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:14:12.944238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:14:12.944238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:12.944238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:12.944238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:12.944238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:12.944238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:12.944238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:12.944238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:12.944238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:12.944238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:12.944238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:12.944238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:12.944238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:12.945332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:12.945332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:12.945332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:12.945332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:12.945332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:14:12.945332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:14:12.945332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:14:12.945332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:14:12.945332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:14:12.945332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:14:12.945332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 19:14:12.945332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:12.948125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 461/921] 10.222.0.17 () {56 vars in 1120 bytes} [Sun Aug 7 19:14:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:12.956077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75babe-ced7-4112-866d-36835aa150a8 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:13.084305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75babe-ced7-4112-866d-36835aa150a8 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:13.124567 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54c43b-7db9-4c80-9eae-7909027d2f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:13.125349 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54c43b-7db9-4c80-9eae-7909027d2f51 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:13.125661 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54c43b-7db9-4c80-9eae-7909027d2f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:13.125934 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54c43b-7db9-4c80-9eae-7909027d2f51 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:13.126619 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54c43b-7db9-4c80-9eae-7909027d2f51 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:13.126927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54c43b-7db9-4c80-9eae-7909027d2f51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:13.128804 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54c43b-7db9-4c80-9eae-7909027d2f51 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:13.138762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 461/922] 10.222.0.17 () {56 vars in 1115 bytes} [Sun Aug 7 19:14:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:13.144375 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa17f9-dd46-4977-8c27-2d1633df59f0 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:13.180393 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:13.256022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa17f9-dd46-4977-8c27-2d1633df59f0 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:13.292991 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631809f-0e16-4854-85b0-f6c9b4e5f5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:13.293657 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631809f-0e16-4854-85b0-f6c9b4e5f5bd None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:13.293937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631809f-0e16-4854-85b0-f6c9b4e5f5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:13.294213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631809f-0e16-4854-85b0-f6c9b4e5f5bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:13.306491 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631809f-0e16-4854-85b0-f6c9b4e5f5bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:13.308463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631809f-0e16-4854-85b0-f6c9b4e5f5bd None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:13.324095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-1631809f-0e16-4854-85b0-f6c9b4e5f5bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:13.324095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:13.324095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:13.324095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:13.324095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:13.324095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:13.324095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:13.324095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:13.324095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:13.324095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:13.324095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:13.324095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:13.324095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:13.324095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:13.324095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:13.324095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:13.325181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:13.325181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:13.325181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:13.325181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:13.325181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:13.325181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:13.325181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:13.325181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:13.325181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:13.325181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:13.325181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:13.325181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:13.325181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:13.325181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:13.325181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:13.325181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:13.325181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:13.326058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:13.326058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:13.326058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:13.326058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 462/923] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:13.332814 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54c43b-7db9-4c80-9eae-7909027d2f51 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:13.392322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:13.451879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54c43b-7db9-4c80-9eae-7909027d2f51 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:13.487653 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197669a7-50cb-44ce-aaab-0674d8bd2094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:13.488315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197669a7-50cb-44ce-aaab-0674d8bd2094 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:13.488592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197669a7-50cb-44ce-aaab-0674d8bd2094 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:13.488863 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197669a7-50cb-44ce-aaab-0674d8bd2094 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:13.489614 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197669a7-50cb-44ce-aaab-0674d8bd2094 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:13.489918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197669a7-50cb-44ce-aaab-0674d8bd2094 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:13.491429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197669a7-50cb-44ce-aaab-0674d8bd2094 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:13.504774 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 462/924] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:13.510722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631809f-0e16-4854-85b0-f6c9b4e5f5bd None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:13.607931 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:13.639184 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631809f-0e16-4854-85b0-f6c9b4e5f5bd None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:13.684859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48881877-5b47-4447-8e3e-68e5e50b24fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:13.685507 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48881877-5b47-4447-8e3e-68e5e50b24fb None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:13.685785 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48881877-5b47-4447-8e3e-68e5e50b24fb None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:13.686058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48881877-5b47-4447-8e3e-68e5e50b24fb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:13.694244 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48881877-5b47-4447-8e3e-68e5e50b24fb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:13.695746 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48881877-5b47-4447-8e3e-68e5e50b24fb None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:13.709206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-48881877-5b47-4447-8e3e-68e5e50b24fb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 19:14:13.709206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:13.709206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:13.709206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:13.709206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:13.709206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:13.709206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:13.709206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:13.709206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:13.709206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:13.709206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:13.709206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:13.709206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:14:13.709206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:14:13.709206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:14:13.709206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:13.710212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:13.711144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:13.711144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:13.711144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:14:13.711144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:14:13.711144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:14:13.711144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:14:13.711144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:14:13.711144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:14:13.711144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 19:14:13.711144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:13.711144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 463/925] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Aug 7 19:14:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:13.716275 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197669a7-50cb-44ce-aaab-0674d8bd2094 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:13.821716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197669a7-50cb-44ce-aaab-0674d8bd2094 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:13.835547 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:13.857283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291380d1-17b8-42cb-9176-2e7c3e794122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:13.857906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291380d1-17b8-42cb-9176-2e7c3e794122 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:13.858185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291380d1-17b8-42cb-9176-2e7c3e794122 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:13.858457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291380d1-17b8-42cb-9176-2e7c3e794122 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:13.859189 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291380d1-17b8-42cb-9176-2e7c3e794122 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:13.859492 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291380d1-17b8-42cb-9176-2e7c3e794122 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:13.860959 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291380d1-17b8-42cb-9176-2e7c3e794122 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:13.879354 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 463/926] 10.222.0.17 () {56 vars in 1149 bytes} [Sun Aug 7 19:14:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:13.885042 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48881877-5b47-4447-8e3e-68e5e50b24fb None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:13.995106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48881877-5b47-4447-8e3e-68e5e50b24fb None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:14.045081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2c2a5-e79b-4402-8747-761df34c45e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:14.045773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2c2a5-e79b-4402-8747-761df34c45e2 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:14.046056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2c2a5-e79b-4402-8747-761df34c45e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:14.046332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2c2a5-e79b-4402-8747-761df34c45e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:14.058798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2c2a5-e79b-4402-8747-761df34c45e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:14.060324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2c2a5-e79b-4402-8747-761df34c45e2 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:14.076856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-25b2c2a5-e79b-4402-8747-761df34c45e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:14.076856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:14.076856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:14.076856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:14.076856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:14.076856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:14.076856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:14.076856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:14.076856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:14.076856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:14.076856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:14.076856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:14.076856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:14.076856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:14.076856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:14.076856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:14.077954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:14.077954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:14.077954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:14.077954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:14.077954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:14.077954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:14.077954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:14.077954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:14.077954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:14.077954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:14.077954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:14.077954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:14.077954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:14.077954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:14.077954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:14.077954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:14.077954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:14.078841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:14.078841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:14.078841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:14.078841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:14.078841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 464/927] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:14.085793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291380d1-17b8-42cb-9176-2e7c3e794122 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:14.192179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291380d1-17b8-42cb-9176-2e7c3e794122 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:14.227424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3089a57-2303-43f0-b5a8-2252bd48f6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:14.228485 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3089a57-2303-43f0-b5a8-2252bd48f6f7 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:14.228985 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3089a57-2303-43f0-b5a8-2252bd48f6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:14.229527 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3089a57-2303-43f0-b5a8-2252bd48f6f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:14.230486 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3089a57-2303-43f0-b5a8-2252bd48f6f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:14.230990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3089a57-2303-43f0-b5a8-2252bd48f6f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:14.232668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3089a57-2303-43f0-b5a8-2252bd48f6f7 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:14.245667 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 464/928] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:14.251673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2c2a5-e79b-4402-8747-761df34c45e2 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:14.275064 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0b958-160a-4aa8-ba51-bd90b4498437 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:14.275682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0b958-160a-4aa8-ba51-bd90b4498437 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:14.276232 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0b958-160a-4aa8-ba51-bd90b4498437 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:14.278144 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 465/929] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:14:14.283485 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ae82e-49c1-4206-bf03-6e43d48d55b7 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:14.284305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ae82e-49c1-4206-bf03-6e43d48d55b7 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:14.284782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ae82e-49c1-4206-bf03-6e43d48d55b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:14.308833 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-c93ae82e-49c1-4206-bf03-6e43d48d55b7 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:14.311057 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-c93ae82e-49c1-4206-bf03-6e43d48d55b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:14.341718 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:14.361743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2c2a5-e79b-4402-8747-761df34c45e2 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:14.387729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 466/930] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 19:14:14.405585 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d0e1b-1acb-4f5d-aaad-7948ed3d8e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:14.406629 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d0e1b-1acb-4f5d-aaad-7948ed3d8e8b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:14.407110 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d0e1b-1acb-4f5d-aaad-7948ed3d8e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:14.407612 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d0e1b-1acb-4f5d-aaad-7948ed3d8e8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:14.410458 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ae82e-49c1-4206-bf03-6e43d48d55b7 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:14.418983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d0e1b-1acb-4f5d-aaad-7948ed3d8e8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:14.421003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d0e1b-1acb-4f5d-aaad-7948ed3d8e8b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:14.434493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-be0d0e1b-1acb-4f5d-aaad-7948ed3d8e8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:14.434493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:14.434493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:14.434493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:14.434493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:14.434493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:14.434493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:14.434493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:14.434493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:14.434493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:14.434493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:14.434493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:14.434493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:14:14.434493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:14:14.434493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:14:14.435466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:14:14.435466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:14.435466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:14.435466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:14.435466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:14.435466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:14.435466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:14.435466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:14.435466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:14.435466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:14.435466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:14.435466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:14.435466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:14.435466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:14.435466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:14.435466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:14.435466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:14.436423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:14:14.436423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:14:14.436423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:14:14.436423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:14:14.436423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:14:14.436423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:14:14.436423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:14.436423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:14.436423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 465/931] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:14:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:14.441708 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d0e1b-1acb-4f5d-aaad-7948ed3d8e8b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:14.500476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:14.532473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ae82e-49c1-4206-bf03-6e43d48d55b7 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:14.567357 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d0e1b-1acb-4f5d-aaad-7948ed3d8e8b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:14.571386 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6c8ff-fb6d-47f0-a864-ad46b3deb6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:14.572325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6c8ff-fb6d-47f0-a864-ad46b3deb6f5 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:14.572810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6c8ff-fb6d-47f0-a864-ad46b3deb6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:14.573295 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6c8ff-fb6d-47f0-a864-ad46b3deb6f5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:14.574152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a6c8ff-fb6d-47f0-a864-ad46b3deb6f5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:14.575767 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a6c8ff-fb6d-47f0-a864-ad46b3deb6f5 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:14.585323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-c0a6c8ff-fb6d-47f0-a864-ad46b3deb6f5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 19:14:14.585323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:14.585323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:14.585323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:14.585323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:14.585323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:14.585323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:14.585323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:14.585323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 19:14:14.585323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:14.585323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:14.585323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:14.585323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:14.585323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:14.585323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:14.585323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:14.585323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:14.586534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:14.586534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:14.586534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:14.586534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:14.586534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:14.586534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:14.586534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:14.586534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:14.586534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:14.586534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:14.586534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:14.586534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:14.586534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:14.586534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:14.586534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:14.586534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:14.587418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:14.587418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:14.587418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:14.587418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:14.587418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:14.587418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:14.587418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:14.587418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:14.587418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:14.587418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:14.587418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:14.587418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:14.587418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 19:14:14.587418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:14.590398 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 467/932] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Aug 7 19:14:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:14.596534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6c8ff-fb6d-47f0-a864-ad46b3deb6f5 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:14.606613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcd803-5ef9-4d43-930d-b266369615e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:14.607728 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcd803-5ef9-4d43-930d-b266369615e9 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:14.608232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcd803-5ef9-4d43-930d-b266369615e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:14.608731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcd803-5ef9-4d43-930d-b266369615e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:14.609776 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcd803-5ef9-4d43-930d-b266369615e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:14.610294 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcd803-5ef9-4d43-930d-b266369615e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:14.612106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcd803-5ef9-4d43-930d-b266369615e9 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:14.626180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:14.633389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 466/933] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:14:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:14.634652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcd803-5ef9-4d43-930d-b266369615e9 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:14.732904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6c8ff-fb6d-47f0-a864-ad46b3deb6f5 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:14.749558 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcd803-5ef9-4d43-930d-b266369615e9 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:14.765923 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:14.780530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5fecd-37d1-422a-b52d-f53df8946927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:14.781156 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5fecd-37d1-422a-b52d-f53df8946927 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:14.781454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5fecd-37d1-422a-b52d-f53df8946927 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:14.781949 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d5fecd-37d1-422a-b52d-f53df8946927 None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:14.782598 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d5fecd-37d1-422a-b52d-f53df8946927 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:14.782911 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d5fecd-37d1-422a-b52d-f53df8946927 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:14.785056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d5fecd-37d1-422a-b52d-f53df8946927 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:14.792442 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b6f95-fef9-4ff4-bf7e-eb9281a22f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:14.793479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b6f95-fef9-4ff4-bf7e-eb9281a22f8e None admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:14.793973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b6f95-fef9-4ff4-bf7e-eb9281a22f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:14.794453 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b6f95-fef9-4ff4-bf7e-eb9281a22f8e None admin] PATH_INFO: `/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/61ff29258a214817b8fc15b51fe8b3e6/roles/c5946f310572439687baeed49acd4336` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:14.796129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 468/934] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Aug 7 19:14:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:14.797055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5fecd-37d1-422a-b52d-f53df8946927 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:14.797533 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b6f95-fef9-4ff4-bf7e-eb9281a22f8e None admin] RBAC: Authorizing `identity:create_grant(project_id=37703d50cd7e4d9bbdc797a7a1466b66, user_id=61ff29258a214817b8fc15b51fe8b3e6, role_id=c5946f310572439687baeed49acd4336)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:14.799272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24b6f95-fef9-4ff4-bf7e-eb9281a22f8e None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:14.825394 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 467/935] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:14:14 2022] PUT /identity/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/61ff29258a214817b8fc15b51fe8b3e6/roles/c5946f310572439687baeed49acd4336 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:14:14.905017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:14.920314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d5fecd-37d1-422a-b52d-f53df8946927 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:14.924547 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae60663-d9f0-4fb2-b5bd-05fa201494c7 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:14.924869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae60663-d9f0-4fb2-b5bd-05fa201494c7 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:14.925104 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae60663-d9f0-4fb2-b5bd-05fa201494c7 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:14.926775 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 468/936] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:14:14.934032 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e518766-07c8-48d8-8a1a-f82cb3045852 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:14.934450 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e518766-07c8-48d8-8a1a-f82cb3045852 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:14.934816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e518766-07c8-48d8-8a1a-f82cb3045852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:14.954340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-9e518766-07c8-48d8-8a1a-f82cb3045852 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:14.955766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9e518766-07c8-48d8-8a1a-f82cb3045852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:14.958424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c57faa-79c3-43cd-9b41-4d4025331536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:14.959095 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c57faa-79c3-43cd-9b41-4d4025331536 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:14.959432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c57faa-79c3-43cd-9b41-4d4025331536 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:14.959713 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c57faa-79c3-43cd-9b41-4d4025331536 None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:14.960382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c57faa-79c3-43cd-9b41-4d4025331536 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:14.960687 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c57faa-79c3-43cd-9b41-4d4025331536 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:14.962092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c57faa-79c3-43cd-9b41-4d4025331536 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:14.968047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 469/937] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Aug 7 19:14:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:14.977380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c57faa-79c3-43cd-9b41-4d4025331536 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:15.042858 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 469/938] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:15.044435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef6436-7f73-4dd6-a836-4cc8d602880b None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:15.044862 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef6436-7f73-4dd6-a836-4cc8d602880b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:15.045290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef6436-7f73-4dd6-a836-4cc8d602880b None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:15.046777 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 470/939] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:15.052809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3411940-9e1b-442c-9808-85a6cd2441e9 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:15.053390 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3411940-9e1b-442c-9808-85a6cd2441e9 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:15.053669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3411940-9e1b-442c-9808-85a6cd2441e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:15.075589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-f3411940-9e1b-442c-9808-85a6cd2441e9 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:15.076815 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f3411940-9e1b-442c-9808-85a6cd2441e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:15.104462 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c57faa-79c3-43cd-9b41-4d4025331536 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:15.140958 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf63507-435d-4b25-8925-3b60f5347475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:15.141605 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf63507-435d-4b25-8925-3b60f5347475 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:15.141882 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf63507-435d-4b25-8925-3b60f5347475 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:15.142261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf63507-435d-4b25-8925-3b60f5347475 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:15.142870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf63507-435d-4b25-8925-3b60f5347475 None admin] RBAC: Adding query filter params (interface=public, service_id=68b5208820074055b91e941db124b2d2, region_id=RegionOne) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:15.143183 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf63507-435d-4b25-8925-3b60f5347475 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:15.144593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf63507-435d-4b25-8925-3b60f5347475 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:15.148393 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 471/940] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 19:14:15.149249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3411940-9e1b-442c-9808-85a6cd2441e9 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:15.156702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 470/941] 10.222.0.17 () {56 vars in 1251 bytes} [Sun Aug 7 19:14:14 2022] GET /identity/v3/endpoints?service_id=68b5208820074055b91e941db124b2d2&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:15.164350 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf63507-435d-4b25-8925-3b60f5347475 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:15.185834 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:15.191239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:15.261861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3411940-9e1b-442c-9808-85a6cd2441e9 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:15.276019 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf63507-435d-4b25-8925-3b60f5347475 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:15.299035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bc7f5-0e32-49d1-8413-1990948691b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:15.299577 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bc7f5-0e32-49d1-8413-1990948691b0 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:15.299854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bc7f5-0e32-49d1-8413-1990948691b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:15.300161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bc7f5-0e32-49d1-8413-1990948691b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:15.300816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bc7f5-0e32-49d1-8413-1990948691b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:15.302288 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bc7f5-0e32-49d1-8413-1990948691b0 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:15.304955 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: WARNING keystone.server.flask.common [None req-532bc7f5-0e32-49d1-8413-1990948691b0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 07 19:14:15.313017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ca661-9dc5-4c55-bb8d-9061ac4c488b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:15.313971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ca661-9dc5-4c55-bb8d-9061ac4c488b None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:15.314450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ca661-9dc5-4c55-bb8d-9061ac4c488b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:15.314900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ca661-9dc5-4c55-bb8d-9061ac4c488b None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:15.315777 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ca661-9dc5-4c55-bb8d-9061ac4c488b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:15.317349 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ca661-9dc5-4c55-bb8d-9061ac4c488b None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:15.323518 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.sql.core [None req-532bc7f5-0e32-49d1-8413-1990948691b0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 07 19:14:15.323518 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 07 19:14:15.323518 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [parameters: {'id': '0c8fe7df664b4be4b8220cec70c407f7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 07 19:14:15.323518 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80138) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 07 19:14:15.332374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 471/942] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Aug 7 19:14:15 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:14:15.332374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ca661-9dc5-4c55-bb8d-9061ac4c488b None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:15.332658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-532bc7f5-0e32-49d1-8413-1990948691b0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0c8fe7df664b4be4b8220cec70c407f7.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0c8fe7df664b4be4b8220cec70c407f7. Aug 07 19:14:15.332658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:15.332658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 19:14:15.332658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 19:14:15.332658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 19:14:15.332658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 19:14:15.332658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 19:14:15.332658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 19:14:15.332658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 19:14:15.332658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application conn.query(q) Aug 07 19:14:15.332658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 19:14:15.332658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 19:14:15.332658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application result.read() Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 07 19:14:15.333415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 07 19:14:15.334360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 19:14:15.334360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application next(self.gen) Aug 07 19:14:15.334360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 07 19:14:15.334360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application yield resource Aug 07 19:14:15.334360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 19:14:15.334360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application next(self.gen) Aug 07 19:14:15.334360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 07 19:14:15.334360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self.session.rollback() Aug 07 19:14:15.334360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 07 19:14:15.334360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self.force_reraise() Aug 07 19:14:15.334360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 07 19:14:15.334360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise self.value Aug 07 19:14:15.334360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 07 19:14:15.334360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 07 19:14:15.334360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 07 19:14:15.334360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application session.commit() Aug 07 19:14:15.334360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 07 19:14:15.335254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 07 19:14:15.335254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 07 19:14:15.335254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._prepare_impl() Aug 07 19:14:15.335254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 07 19:14:15.335254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self.session.flush() Aug 07 19:14:15.335254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 07 19:14:15.335254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._flush(objects) Aug 07 19:14:15.335254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 07 19:14:15.335254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 07 19:14:15.335254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 07 19:14:15.335254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application compat.raise_( Aug 07 19:14:15.335254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 19:14:15.335254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception Aug 07 19:14:15.335254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 07 19:14:15.335254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application flush_context.execute() Aug 07 19:14:15.335254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 07 19:14:15.335254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rec.execute(self) Aug 07 19:14:15.337062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 07 19:14:15.337062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 07 19:14:15.337062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 07 19:14:15.337062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 07 19:14:15.337062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 07 19:14:15.337062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 07 19:14:15.337062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 07 19:14:15.337062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 07 19:14:15.337062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 07 19:14:15.337062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 07 19:14:15.337062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 07 19:14:15.337062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 07 19:14:15.337062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 07 19:14:15.337062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 07 19:14:15.337062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 07 19:14:15.338905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 07 19:14:15.338905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 19:14:15.338905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception Aug 07 19:14:15.338905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 19:14:15.338905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 19:14:15.338905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 19:14:15.338905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 19:14:15.338905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 19:14:15.338905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 19:14:15.338905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 19:14:15.338905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application conn.query(q) Aug 07 19:14:15.338905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 19:14:15.338905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 19:14:15.338905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 19:14:15.338905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application result.read() Aug 07 19:14:15.338905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 19:14:15.338905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 19:14:15.345737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 19:14:15.345737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 19:14:15.345737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 19:14:15.345737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 19:14:15.345737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 19:14:15.345737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 19:14:15.345737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 07 19:14:15.345737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 07 19:14:15.345737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [parameters: {'id': '0c8fe7df664b4be4b8220cec70c407f7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 07 19:14:15.345737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 07 19:14:15.345737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:15.345737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:15.345737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:15.345737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:15.345737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 07 19:14:15.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 07 19:14:15.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 07 19:14:15.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 07 19:14:15.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 07 19:14:15.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:15.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:15.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:15.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:15.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:15.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:15.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:15.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:15.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:15.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:15.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:15.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:15.346707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:15.347856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:15.347856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 07 19:14:15.347856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 07 19:14:15.347856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:15.347856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:15.347856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 07 19:14:15.347856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 07 19:14:15.347856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0c8fe7df664b4be4b8220cec70c407f7. Aug 07 19:14:15.347856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:15.347856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 472/943] 10.222.0.17 () {60 vars in 1140 bytes} [Sun Aug 7 19:14:15 2022] POST /identity/v3/projects => generated 262 bytes in 184 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:15.348711 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bc7f5-0e32-49d1-8413-1990948691b0 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:15.434854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:15.453036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:15.457838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ca661-9dc5-4c55-bb8d-9061ac4c488b None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:15.465641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bc7f5-0e32-49d1-8413-1990948691b0 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:15.501986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726177a-1985-4818-82ec-18efc1e026e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:15.503363 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726177a-1985-4818-82ec-18efc1e026e0 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:15.503363 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726177a-1985-4818-82ec-18efc1e026e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:15.503363 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726177a-1985-4818-82ec-18efc1e026e0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:15.503785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726177a-1985-4818-82ec-18efc1e026e0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:15.505241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726177a-1985-4818-82ec-18efc1e026e0 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:15.507553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50659745-1547-48b6-9425-c8ac538ef6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:15.508626 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50659745-1547-48b6-9425-c8ac538ef6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:15.509103 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50659745-1547-48b6-9425-c8ac538ef6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:15.509564 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50659745-1547-48b6-9425-c8ac538ef6e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:15.516031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-1726177a-1985-4818-82ec-18efc1e026e0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 19:14:15.516031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:15.516031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:15.516031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:15.516031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:15.516031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:15.516031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:15.516031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:15.516031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 19:14:15.516031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:15.516031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:15.516031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:15.516031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:15.516031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:15.516031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:15.516031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:15.517230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:15.517230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:15.517230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:15.517230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:15.517230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:15.517230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:15.517230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:15.517230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:15.517230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:15.517230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:15.517230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:15.517230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:15.517230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:15.517230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:15.517230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:15.517230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:15.517230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:15.518323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:15.518323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:15.518323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:15.518323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:15.518323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:15.518323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:15.518323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:15.518323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:15.518323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:15.518323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:15.518323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:15.518323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:15.518323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 19:14:15.518323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:15.518323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 472/944] 10.222.0.17 () {56 vars in 1132 bytes} [Sun Aug 7 19:14:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:15.522310 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50659745-1547-48b6-9425-c8ac538ef6e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:15.524119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50659745-1547-48b6-9425-c8ac538ef6e4 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:15.525923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726177a-1985-4818-82ec-18efc1e026e0 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:15.537191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-50659745-1547-48b6-9425-c8ac538ef6e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:15.537191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:15.537191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:15.537191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:15.537191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:15.537191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:15.537191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:15.537191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:15.537191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:15.537191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:15.537191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:15.537191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:15.537191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:14:15.537191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:14:15.537191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:14:15.538321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:14:15.538321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:15.538321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:15.538321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:15.538321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:15.538321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:15.538321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:15.538321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:15.538321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:15.538321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:15.538321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:15.538321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:15.538321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:15.538321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:15.538321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:15.538321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:15.538321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:15.539434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:14:15.539434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:14:15.539434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:14:15.539434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:14:15.539434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:14:15.539434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:14:15.539434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:15.539434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:15.540504 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 473/945] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:14:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:15.546185 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50659745-1547-48b6-9425-c8ac538ef6e4 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:15.661169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50659745-1547-48b6-9425-c8ac538ef6e4 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:15.672528 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726177a-1985-4818-82ec-18efc1e026e0 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:15.708579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e95fa-c627-42b2-8e9d-51c5a6f031eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:15.709207 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e95fa-c627-42b2-8e9d-51c5a6f031eb None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:15.709465 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e95fa-c627-42b2-8e9d-51c5a6f031eb None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:15.709741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e95fa-c627-42b2-8e9d-51c5a6f031eb None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:15.710144 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160a77a-04a3-457c-9cad-b27a33295dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:15.710414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e95fa-c627-42b2-8e9d-51c5a6f031eb None admin] RBAC: Adding query filter params (name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:15.710716 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e95fa-c627-42b2-8e9d-51c5a6f031eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:15.710883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160a77a-04a3-457c-9cad-b27a33295dae None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:15.711039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160a77a-04a3-457c-9cad-b27a33295dae None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:15.716045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160a77a-04a3-457c-9cad-b27a33295dae None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:15.716045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160a77a-04a3-457c-9cad-b27a33295dae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:15.716313 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e95fa-c627-42b2-8e9d-51c5a6f031eb None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:15.716420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160a77a-04a3-457c-9cad-b27a33295dae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:15.716420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e160a77a-04a3-457c-9cad-b27a33295dae None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:15.720256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 473/946] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Aug 7 19:14:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:15.732702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160a77a-04a3-457c-9cad-b27a33295dae None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:15.737467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 474/947] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Aug 7 19:14:15 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:15.740194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:15.847613 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e160a77a-04a3-457c-9cad-b27a33295dae None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:15.890917 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8e952-5746-404e-9c1f-6c9ffa6beb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:15.891554 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8e952-5746-404e-9c1f-6c9ffa6beb9a None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:15.891833 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8e952-5746-404e-9c1f-6c9ffa6beb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:15.892136 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8e952-5746-404e-9c1f-6c9ffa6beb9a None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:15.892785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8e952-5746-404e-9c1f-6c9ffa6beb9a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:15.893089 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8e952-5746-404e-9c1f-6c9ffa6beb9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:15.894493 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8e952-5746-404e-9c1f-6c9ffa6beb9a None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:15.900664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 474/948] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Aug 7 19:14:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:15.907788 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e95fa-c627-42b2-8e9d-51c5a6f031eb None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:15.930854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:16.032034 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e95fa-c627-42b2-8e9d-51c5a6f031eb None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:16.062082 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed4730-4c7a-4c77-ac2e-002f0626bc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:16.062708 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed4730-4c7a-4c77-ac2e-002f0626bc49 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:16.062983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed4730-4c7a-4c77-ac2e-002f0626bc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:16.063251 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed4730-4c7a-4c77-ac2e-002f0626bc49 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:16.063919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed4730-4c7a-4c77-ac2e-002f0626bc49 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:16.065340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed4730-4c7a-4c77-ac2e-002f0626bc49 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:16.089625 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 475/949] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:14:15 2022] POST /identity/v3/endpoints => generated 347 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:14:16.949199 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3146e81-b95d-40dd-b53f-6f732ab11668 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:16.950108 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3146e81-b95d-40dd-b53f-6f732ab11668 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:16.950559 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3146e81-b95d-40dd-b53f-6f732ab11668 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:16.952565 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 475/950] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:16.959066 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e4716-ea86-49ba-a7de-288c1cf68079 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:16.959981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e4716-ea86-49ba-a7de-288c1cf68079 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:16.960453 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e4716-ea86-49ba-a7de-288c1cf68079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:16.979801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-7a2e4716-ea86-49ba-a7de-288c1cf68079 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:16.981725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-7a2e4716-ea86-49ba-a7de-288c1cf68079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:17.056673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:17.080146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 476/951] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 19:14:17.097306 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3146e81-b95d-40dd-b53f-6f732ab11668 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:17.170158 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:17.203523 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3146e81-b95d-40dd-b53f-6f732ab11668 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:17.244566 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d18c4a-c71f-4824-9e7d-e737074228d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:17.245220 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d18c4a-c71f-4824-9e7d-e737074228d1 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:17.245548 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d18c4a-c71f-4824-9e7d-e737074228d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:17.245819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d18c4a-c71f-4824-9e7d-e737074228d1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:17.256490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d18c4a-c71f-4824-9e7d-e737074228d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:14:17.261207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-26d18c4a-c71f-4824-9e7d-e737074228d1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 19:14:17.261207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:17.261207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:17.261207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:17.261207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:17.261207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:17.261207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:17.261207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:17.261207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:17.261207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:17.261207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:17.261207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:17.261207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:14:17.261207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:14:17.261207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:14:17.261207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:14:17.262130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:14:17.262130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:14:17.262130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:14:17.262130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:14:17.262130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:17.262130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:17.262130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:17.262130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:17.262130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:17.262130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:17.262130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:17.262130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:17.262130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:17.262130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:17.262130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:17.262130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:17.263365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:17.263365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:17.263365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:17.263365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:17.263365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:14:17.263365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:14:17.263365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:14:17.263365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:14:17.263365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:14:17.263365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:14:17.263365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 19:14:17.263365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:17.263365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 476/952] 10.222.0.17 () {56 vars in 1120 bytes} [Sun Aug 7 19:14:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:17.270745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e4716-ea86-49ba-a7de-288c1cf68079 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:17.377714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e4716-ea86-49ba-a7de-288c1cf68079 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:17.412647 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014da647-829f-480a-ac05-8f42f2e42ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:17.413461 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014da647-829f-480a-ac05-8f42f2e42ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:17.413739 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014da647-829f-480a-ac05-8f42f2e42ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:17.414009 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014da647-829f-480a-ac05-8f42f2e42ab1 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:17.414731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014da647-829f-480a-ac05-8f42f2e42ab1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:17.415033 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014da647-829f-480a-ac05-8f42f2e42ab1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:17.416456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014da647-829f-480a-ac05-8f42f2e42ab1 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:17.426590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 477/953] 10.222.0.17 () {56 vars in 1115 bytes} [Sun Aug 7 19:14:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:17.432742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d18c4a-c71f-4824-9e7d-e737074228d1 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:17.458769 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e840cd3-73fa-4af0-ac58-4ac5097e428d None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:17.459064 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e840cd3-73fa-4af0-ac58-4ac5097e428d None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:17.459259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e840cd3-73fa-4af0-ac58-4ac5097e428d None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:17.461020 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 478/954] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:14:17.466092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadfbaf-34da-4062-932c-6d9a9041c289 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:17.466375 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadfbaf-34da-4062-932c-6d9a9041c289 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:17.466647 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadfbaf-34da-4062-932c-6d9a9041c289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:17.484876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-8cadfbaf-34da-4062-932c-6d9a9041c289 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:17.486069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8cadfbaf-34da-4062-932c-6d9a9041c289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:17.537351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:17.544296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d18c4a-c71f-4824-9e7d-e737074228d1 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:17.564342 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 479/955] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 19:14:17.578216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:17.582385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadfbaf-34da-4062-932c-6d9a9041c289 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:17.584731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f53462-b135-4605-9851-c8a9f352d930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:17.585737 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f53462-b135-4605-9851-c8a9f352d930 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:17.586235 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f53462-b135-4605-9851-c8a9f352d930 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:17.586726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f53462-b135-4605-9851-c8a9f352d930 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:17.599043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f53462-b135-4605-9851-c8a9f352d930 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:17.601021 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f53462-b135-4605-9851-c8a9f352d930 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:17.617550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-93f53462-b135-4605-9851-c8a9f352d930 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:17.617550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:17.617550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:17.617550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:17.617550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:17.617550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:17.617550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:17.617550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:17.617550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:17.617550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:17.617550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:17.617550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:17.617550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:17.617550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:17.617550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:17.617550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:17.618602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:17.618602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:17.618602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:17.618602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:17.618602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:17.618602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:17.618602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:17.618602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:17.618602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:17.618602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:17.618602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:17.618602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:17.618602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:17.618602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:17.618602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:17.618602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:17.618602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:17.619623 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:17.619623 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:17.619623 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:17.626359 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 477/956] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:17.627920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f53462-b135-4605-9851-c8a9f352d930 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:17.687929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadfbaf-34da-4062-932c-6d9a9041c289 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:17.727923 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d3565-d870-4afe-93f9-252a2e097e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:17.728517 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d3565-d870-4afe-93f9-252a2e097e1b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:17.728792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d3565-d870-4afe-93f9-252a2e097e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:17.729058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d3565-d870-4afe-93f9-252a2e097e1b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:17.729680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d3565-d870-4afe-93f9-252a2e097e1b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:17.731128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d3565-d870-4afe-93f9-252a2e097e1b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:17.741284 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f53462-b135-4605-9851-c8a9f352d930 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:17.742298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-f85d3565-d870-4afe-93f9-252a2e097e1b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 19:14:17.742298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:17.742298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:17.742298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:17.742298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:17.742298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:17.742298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:17.742298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:17.742298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 19:14:17.742298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:17.742298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:17.742298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:17.742298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:17.742298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:17.742298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:17.742298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:17.742298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:17.743592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:17.743592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:17.743592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:17.743592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:17.743592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:17.743592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:17.743592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:17.743592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:17.743592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:17.743592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:17.743592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:17.743592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:17.743592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:17.743592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:17.743592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:17.743592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:17.744758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:17.744758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:17.744758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:17.744758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:17.744758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:17.744758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:17.744758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:17.744758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:17.744758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:17.744758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:17.744758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:17.744758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:17.744758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 19:14:17.744758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:17.748614 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 480/957] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Aug 7 19:14:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:17.749642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d3565-d870-4afe-93f9-252a2e097e1b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:17.783232 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739df43-ddae-408e-9d88-41230d4ef876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:17.784229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739df43-ddae-408e-9d88-41230d4ef876 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:17.784657 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739df43-ddae-408e-9d88-41230d4ef876 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:17.785176 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739df43-ddae-408e-9d88-41230d4ef876 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:17.785972 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739df43-ddae-408e-9d88-41230d4ef876 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:17.786420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739df43-ddae-408e-9d88-41230d4ef876 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:17.788013 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739df43-ddae-408e-9d88-41230d4ef876 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:17.799465 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:17.804401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 478/958] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:17.810872 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739df43-ddae-408e-9d88-41230d4ef876 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:17.860944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d3565-d870-4afe-93f9-252a2e097e1b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:17.888377 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:17.903371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ada105-9093-4c2b-942a-af771e2cfb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:17.904500 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ada105-9093-4c2b-942a-af771e2cfb1d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:17.904781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ada105-9093-4c2b-942a-af771e2cfb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:17.905058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ada105-9093-4c2b-942a-af771e2cfb1d None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:17.905822 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ada105-9093-4c2b-942a-af771e2cfb1d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:17.906201 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ada105-9093-4c2b-942a-af771e2cfb1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:17.908558 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ada105-9093-4c2b-942a-af771e2cfb1d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:17.916106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 481/959] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Aug 7 19:14:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:17.921209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ada105-9093-4c2b-942a-af771e2cfb1d None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:17.939340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739df43-ddae-408e-9d88-41230d4ef876 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:17.980215 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a202ba29-7492-47f2-b34c-0c627442b0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:17.980845 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202ba29-7492-47f2-b34c-0c627442b0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:17.981364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202ba29-7492-47f2-b34c-0c627442b0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:17.981364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a202ba29-7492-47f2-b34c-0c627442b0a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:17.989843 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202ba29-7492-47f2-b34c-0c627442b0a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:17.991320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a202ba29-7492-47f2-b34c-0c627442b0a1 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:18.004766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-a202ba29-7492-47f2-b34c-0c627442b0a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 19:14:18.004766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:18.004766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:18.004766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:18.004766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:18.004766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:18.004766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:18.004766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:18.004766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:18.004766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:18.004766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:18.004766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:18.004766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:14:18.004766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:14:18.004766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:14:18.004766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:18.005790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:18.006754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:18.006754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:18.006754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:14:18.006754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:14:18.006754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:14:18.006754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:14:18.006754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:14:18.006754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:14:18.006754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 19:14:18.006754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:18.012030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 479/960] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Aug 7 19:14:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:18.013277 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e57acf-1ca4-4494-b705-fa86891a2c4d None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:18.013568 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e57acf-1ca4-4494-b705-fa86891a2c4d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:18.013845 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e57acf-1ca4-4494-b705-fa86891a2c4d None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:18.016889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 480/961] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:18.017837 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e57acf-1ca4-4494-b705-fa86891a2c4d None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:18.045379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ada105-9093-4c2b-942a-af771e2cfb1d None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:18.059667 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:18.085924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7dd00-4dab-48ff-af93-32e456a2feae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:18.086554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7dd00-4dab-48ff-af93-32e456a2feae None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:18.086837 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7dd00-4dab-48ff-af93-32e456a2feae None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:18.087111 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7dd00-4dab-48ff-af93-32e456a2feae None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:18.087748 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7dd00-4dab-48ff-af93-32e456a2feae None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:18.088088 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7dd00-4dab-48ff-af93-32e456a2feae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:18.089632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7dd00-4dab-48ff-af93-32e456a2feae None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:18.095107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 482/962] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Aug 7 19:14:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:18.096614 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2415d-f046-46ed-9006-1a1afe9f4e45 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:18.097132 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2415d-f046-46ed-9006-1a1afe9f4e45 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:18.097604 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2415d-f046-46ed-9006-1a1afe9f4e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:18.115698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-a9a2415d-f046-46ed-9006-1a1afe9f4e45 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:18.116914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a9a2415d-f046-46ed-9006-1a1afe9f4e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:18.134797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e57acf-1ca4-4494-b705-fa86891a2c4d None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:18.137431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:18.169862 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ca5a3-d238-4418-89cf-6bee0b6aed36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:18.170476 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ca5a3-d238-4418-89cf-6bee0b6aed36 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:18.170757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ca5a3-d238-4418-89cf-6bee0b6aed36 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:18.171033 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ca5a3-d238-4418-89cf-6bee0b6aed36 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:18.171765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ca5a3-d238-4418-89cf-6bee0b6aed36 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:18.172109 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ca5a3-d238-4418-89cf-6bee0b6aed36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:18.173535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ca5a3-d238-4418-89cf-6bee0b6aed36 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:18.184939 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 483/963] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:18.185773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2415d-f046-46ed-9006-1a1afe9f4e45 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:18.192583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 481/964] 10.222.0.17 () {56 vars in 1149 bytes} [Sun Aug 7 19:14:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:18.194120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504708fa-1d56-4a2d-8d36-6a48627e4e54 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:18.194478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504708fa-1d56-4a2d-8d36-6a48627e4e54 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:18.194829 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504708fa-1d56-4a2d-8d36-6a48627e4e54 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:18.196167 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 482/965] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:18.197558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504708fa-1d56-4a2d-8d36-6a48627e4e54 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:18.299723 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2415d-f046-46ed-9006-1a1afe9f4e45 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:18.304493 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504708fa-1d56-4a2d-8d36-6a48627e4e54 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:18.308369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:18.341853 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5637091a-1e2b-4fd3-874d-68525d9cc190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:18.343272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637091a-1e2b-4fd3-874d-68525d9cc190 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:18.343272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637091a-1e2b-4fd3-874d-68525d9cc190 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:18.343272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5637091a-1e2b-4fd3-874d-68525d9cc190 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:18.343802 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef085051-9ac0-45d5-bb01-52b558e0a676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:18.343920 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637091a-1e2b-4fd3-874d-68525d9cc190 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:18.344463 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef085051-9ac0-45d5-bb01-52b558e0a676 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:18.344747 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef085051-9ac0-45d5-bb01-52b558e0a676 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:18.345009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef085051-9ac0-45d5-bb01-52b558e0a676 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:18.345239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5637091a-1e2b-4fd3-874d-68525d9cc190 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:18.370114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 484/966] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:14:18 2022] POST /identity/v3/endpoints => generated 349 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:14:18.371456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeaede8-2a10-412d-97d0-c0de38123f05 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:18.371731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeaede8-2a10-412d-97d0-c0de38123f05 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:18.372030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeaede8-2a10-412d-97d0-c0de38123f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:18.380864 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:18.417681 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-cfeaede8-2a10-412d-97d0-c0de38123f05 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:18.418943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-cfeaede8-2a10-412d-97d0-c0de38123f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:18.435079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef085051-9ac0-45d5-bb01-52b558e0a676 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:18.436610 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef085051-9ac0-45d5-bb01-52b558e0a676 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:18.512554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 485/967] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:18.513765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeaede8-2a10-412d-97d0-c0de38123f05 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:18.570688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 483/968] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:14:18 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:14:18.571331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef085051-9ac0-45d5-bb01-52b558e0a676 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:18.612678 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:18.627383 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeaede8-2a10-412d-97d0-c0de38123f05 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:18.637397 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:18.666799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee50e4b-1cbe-47dc-b341-83426689d5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:18.667474 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50e4b-1cbe-47dc-b341-83426689d5f8 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:18.667747 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50e4b-1cbe-47dc-b341-83426689d5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:18.668036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50e4b-1cbe-47dc-b341-83426689d5f8 None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:18.668669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee50e4b-1cbe-47dc-b341-83426689d5f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:18.670252 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee50e4b-1cbe-47dc-b341-83426689d5f8 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:18.678941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef085051-9ac0-45d5-bb01-52b558e0a676 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:18.687040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.sql.core [None req-eee50e4b-1cbe-47dc-b341-83426689d5f8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 07 19:14:18.687040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 07 19:14:18.687040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [parameters: {'user_id': '8931599825434f1aab2e3a42cdb73161', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 07 19:14:18.687040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80138) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 07 19:14:18.694599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-eee50e4b-1cbe-47dc-b341-83426689d5f8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 07 19:14:18.694599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:18.694599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 19:14:18.694599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 19:14:18.694599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 19:14:18.694599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 19:14:18.694599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 19:14:18.694599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 19:14:18.694599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 19:14:18.694599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application conn.query(q) Aug 07 19:14:18.694599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 19:14:18.694599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 19:14:18.694599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 19:14:18.694599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application result.read() Aug 07 19:14:18.694599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 19:14:18.695539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application next(self.gen) Aug 07 19:14:18.696880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 07 19:14:18.696880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application yield resource Aug 07 19:14:18.696880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 19:14:18.696880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application next(self.gen) Aug 07 19:14:18.696880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 07 19:14:18.696880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self.session.rollback() Aug 07 19:14:18.696880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 07 19:14:18.696880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self.force_reraise() Aug 07 19:14:18.696880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 07 19:14:18.696880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise self.value Aug 07 19:14:18.696880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 07 19:14:18.696880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 07 19:14:18.696880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 07 19:14:18.696880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application session.commit() Aug 07 19:14:18.696880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 07 19:14:18.696880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 07 19:14:18.696880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 07 19:14:18.697812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._prepare_impl() Aug 07 19:14:18.697812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 07 19:14:18.697812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self.session.flush() Aug 07 19:14:18.697812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 07 19:14:18.697812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._flush(objects) Aug 07 19:14:18.697812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 07 19:14:18.697812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 07 19:14:18.697812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 07 19:14:18.697812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application compat.raise_( Aug 07 19:14:18.697812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 19:14:18.697812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception Aug 07 19:14:18.697812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 07 19:14:18.697812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application flush_context.execute() Aug 07 19:14:18.697812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 07 19:14:18.697812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rec.execute(self) Aug 07 19:14:18.697812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 07 19:14:18.697812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 07 19:14:18.698845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 07 19:14:18.698845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 07 19:14:18.698845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 07 19:14:18.698845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 07 19:14:18.698845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 07 19:14:18.698845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 07 19:14:18.698845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 07 19:14:18.698845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 07 19:14:18.698845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 07 19:14:18.698845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 07 19:14:18.698845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 07 19:14:18.698845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 07 19:14:18.698845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 07 19:14:18.698845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 07 19:14:18.698845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 19:14:18.698845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception Aug 07 19:14:18.699697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 19:14:18.699697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 19:14:18.699697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 19:14:18.699697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 19:14:18.699697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 19:14:18.699697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 19:14:18.699697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 19:14:18.699697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application conn.query(q) Aug 07 19:14:18.699697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 19:14:18.699697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 19:14:18.699697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 19:14:18.699697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application result.read() Aug 07 19:14:18.699697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 19:14:18.699697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 19:14:18.699697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 19:14:18.699697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 19:14:18.700608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 19:14:18.700608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 19:14:18.700608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 19:14:18.700608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 19:14:18.700608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 07 19:14:18.700608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 07 19:14:18.700608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [parameters: {'user_id': '8931599825434f1aab2e3a42cdb73161', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 07 19:14:18.700608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 07 19:14:18.700608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:18.700608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:18.700608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:18.700608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:18.700608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:18.700608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:18.700608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:18.702755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:18.702755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:18.702755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:18.702755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:18.702755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:18.702755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:18.702755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:18.702755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 07 19:14:18.702755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 07 19:14:18.702755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:18.702755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:18.702755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:14:18.702755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:18.702755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:14:18.702755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:18.702755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 07 19:14:18.702755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 07 19:14:18.703919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 07 19:14:18.703919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 07 19:14:18.703919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 07 19:14:18.703919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 07 19:14:18.703919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 07 19:14:18.703919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:18.703919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 486/969] 10.222.0.17 () {60 vars in 1131 bytes} [Sun Aug 7 19:14:18 2022] POST /identity/v3/users => generated 160 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:18.703919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee50e4b-1cbe-47dc-b341-83426689d5f8 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:18.725730 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db3020-17d2-4f85-b9f3-1f7d8cd6ceac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:18.726378 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db3020-17d2-4f85-b9f3-1f7d8cd6ceac None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:18.726655 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db3020-17d2-4f85-b9f3-1f7d8cd6ceac None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:18.726928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db3020-17d2-4f85-b9f3-1f7d8cd6ceac None admin] PATH_INFO: `/v3/services/compute` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:18.727512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db3020-17d2-4f85-b9f3-1f7d8cd6ceac None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:18.729043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db3020-17d2-4f85-b9f3-1f7d8cd6ceac None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:18.738248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-39db3020-17d2-4f85-b9f3-1f7d8cd6ceac None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 19:14:18.738248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:18.738248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:18.738248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:18.738248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:18.738248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:18.738248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:18.738248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:18.738248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 19:14:18.738248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:18.738248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:18.738248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:18.738248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:18.738248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:18.738248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:18.738248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:18.738248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:18.739302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:18.739302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:18.739302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:18.739302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:18.739302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:18.739302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:18.739302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:18.739302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:18.739302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:18.739302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:18.739302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:18.739302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:18.739302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:18.739302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:18.739302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:18.739302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:18.740164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:18.740164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:18.740164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:18.740164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:18.740164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:18.740164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:18.740164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:18.740164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:18.740164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:18.740164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:18.740164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:18.740164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:18.740164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 19:14:18.740164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:18.740164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 484/970] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:14:18 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:18.741021 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db3020-17d2-4f85-b9f3-1f7d8cd6ceac None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:18.814530 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee50e4b-1cbe-47dc-b341-83426689d5f8 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:18.851006 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db3020-17d2-4f85-b9f3-1f7d8cd6ceac None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:18.852022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db155d1-530f-47a6-a107-d98ba9669182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:18.852693 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db155d1-530f-47a6-a107-d98ba9669182 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:18.852973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db155d1-530f-47a6-a107-d98ba9669182 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:18.853250 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db155d1-530f-47a6-a107-d98ba9669182 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:18.865146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db155d1-530f-47a6-a107-d98ba9669182 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:18.866859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db155d1-530f-47a6-a107-d98ba9669182 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:18.881444 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:18.882437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-8db155d1-530f-47a6-a107-d98ba9669182 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:18.882437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:18.882437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:18.882437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:18.882437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:18.882437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:18.882437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:18.882437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:18.882437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:18.882437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:18.882437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:18.882437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:18.882437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:18.882437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:18.882437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:18.882437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:18.883454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:18.883454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:18.883454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:18.883454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:18.883454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:18.883454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:18.883454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:18.883454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:18.883454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:18.883454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:18.883454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:18.883454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:18.883454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:18.883454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:18.883454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:18.883454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:18.883454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:18.884443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:18.884443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:18.884443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:18.884443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 487/971] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:18.885233 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db155d1-530f-47a6-a107-d98ba9669182 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:18.892532 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:18.898599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac348775-07da-4f52-ab84-706b9f38eb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:18.899255 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac348775-07da-4f52-ab84-706b9f38eb27 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:18.899537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac348775-07da-4f52-ab84-706b9f38eb27 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:18.899839 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac348775-07da-4f52-ab84-706b9f38eb27 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:18.908299 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac348775-07da-4f52-ab84-706b9f38eb27 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:18.910462 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac348775-07da-4f52-ab84-706b9f38eb27 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:18.925883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-ac348775-07da-4f52-ab84-706b9f38eb27 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 19:14:18.925883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:18.925883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:18.925883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:18.925883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:18.925883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:18.925883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:18.925883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:18.925883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:18.925883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:18.925883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:18.925883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:18.925883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:14:18.925883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:14:18.925883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:14:18.925883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:18.926884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:18.927838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:18.927838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:18.927838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:14:18.927838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:14:18.927838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:14:18.927838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:14:18.927838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:14:18.927838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:14:18.927838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 19:14:18.927838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:18.932680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 485/972] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:14:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:18.933716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac348775-07da-4f52-ab84-706b9f38eb27 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:18.999596 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db155d1-530f-47a6-a107-d98ba9669182 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:19.040570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede1028-2d62-4487-a255-555b4a8208ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:19.041230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede1028-2d62-4487-a255-555b4a8208ad None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:19.041466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede1028-2d62-4487-a255-555b4a8208ad None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:19.041738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede1028-2d62-4487-a255-555b4a8208ad None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:19.042419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede1028-2d62-4487-a255-555b4a8208ad None admin] RBAC: Adding query filter params (name=compute) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:19.042728 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede1028-2d62-4487-a255-555b4a8208ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:19.043067 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac348775-07da-4f52-ab84-706b9f38eb27 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:19.044166 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede1028-2d62-4487-a255-555b4a8208ad None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:19.052609 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 488/973] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Aug 7 19:14:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:19.058306 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede1028-2d62-4487-a255-555b4a8208ad None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:19.080835 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d29cc5-1cd4-452d-a8c0-00dec5f6577f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:19.081457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d29cc5-1cd4-452d-a8c0-00dec5f6577f None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:19.082035 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d29cc5-1cd4-452d-a8c0-00dec5f6577f None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:19.082035 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d29cc5-1cd4-452d-a8c0-00dec5f6577f None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:19.082647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d29cc5-1cd4-452d-a8c0-00dec5f6577f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:19.082954 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d29cc5-1cd4-452d-a8c0-00dec5f6577f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:19.084471 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d29cc5-1cd4-452d-a8c0-00dec5f6577f None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:19.098745 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 486/974] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:19.099429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d29cc5-1cd4-452d-a8c0-00dec5f6577f None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:19.132698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:19.161244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:19.186560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede1028-2d62-4487-a255-555b4a8208ad None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:19.216175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d29cc5-1cd4-452d-a8c0-00dec5f6577f None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:19.222397 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6895d3f-74e6-4daf-82b9-251c18f0c3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:19.223022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6895d3f-74e6-4daf-82b9-251c18f0c3a7 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:19.223300 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6895d3f-74e6-4daf-82b9-251c18f0c3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:19.223565 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6895d3f-74e6-4daf-82b9-251c18f0c3a7 None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:19.224430 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6895d3f-74e6-4daf-82b9-251c18f0c3a7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:19.224736 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6895d3f-74e6-4daf-82b9-251c18f0c3a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:19.226309 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6895d3f-74e6-4daf-82b9-251c18f0c3a7 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:19.251279 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8edd8-f1dc-4ebc-ad1d-7f1fa37e4fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:19.251895 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8edd8-f1dc-4ebc-ad1d-7f1fa37e4fde None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:19.252214 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8edd8-f1dc-4ebc-ad1d-7f1fa37e4fde None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:19.252485 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8edd8-f1dc-4ebc-ad1d-7f1fa37e4fde None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:19.253115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8edd8-f1dc-4ebc-ad1d-7f1fa37e4fde None admin] RBAC: Adding query filter params (type=compute) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:19.253417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8edd8-f1dc-4ebc-ad1d-7f1fa37e4fde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:19.254801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f8edd8-f1dc-4ebc-ad1d-7f1fa37e4fde None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:19.259404 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 487/975] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Aug 7 19:14:19 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:19.260232 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8edd8-f1dc-4ebc-ad1d-7f1fa37e4fde None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:19.275138 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 489/976] 10.222.0.17 () {56 vars in 1101 bytes} [Sun Aug 7 19:14:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:19.395354 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8edd8-f1dc-4ebc-ad1d-7f1fa37e4fde None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:19.408925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:19.439763 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11cd4f-4ea6-497c-a4a0-2663a313dd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:19.440828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11cd4f-4ea6-497c-a4a0-2663a313dd57 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:19.441306 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11cd4f-4ea6-497c-a4a0-2663a313dd57 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:19.441797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11cd4f-4ea6-497c-a4a0-2663a313dd57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:19.450910 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11cd4f-4ea6-497c-a4a0-2663a313dd57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:19.452911 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11cd4f-4ea6-497c-a4a0-2663a313dd57 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:19.466679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-de11cd4f-4ea6-497c-a4a0-2663a313dd57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:19.466679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:19.466679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:19.466679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:19.466679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:19.466679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:19.466679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:19.466679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:19.466679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:19.466679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:19.466679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:19.466679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:19.466679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:14:19.466679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:14:19.466679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:14:19.467650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:14:19.467650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:19.467650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:19.467650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:19.467650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:19.467650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:19.467650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:19.467650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:19.467650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:19.467650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:19.467650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:19.467650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:19.467650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:19.467650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:19.467650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:19.467650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:19.467650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:19.468629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:14:19.468629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:14:19.468629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:14:19.468629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:14:19.468629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:14:19.468629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:14:19.468629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:19.468629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:19.468629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 488/977] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:14:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:19.474040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6895d3f-74e6-4daf-82b9-251c18f0c3a7 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:19.593632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6895d3f-74e6-4daf-82b9-251c18f0c3a7 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:19.618924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:19.630868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b814187-948a-423f-86dd-af4c2afc399f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:19.631483 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b814187-948a-423f-86dd-af4c2afc399f None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:19.631756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b814187-948a-423f-86dd-af4c2afc399f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:19.632048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b814187-948a-423f-86dd-af4c2afc399f None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:19.632741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b814187-948a-423f-86dd-af4c2afc399f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:19.633040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b814187-948a-423f-86dd-af4c2afc399f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:19.634470 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b814187-948a-423f-86dd-af4c2afc399f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:19.646980 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 490/978] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:14:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:19.652999 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11cd4f-4ea6-497c-a4a0-2663a313dd57 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:19.757848 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11cd4f-4ea6-497c-a4a0-2663a313dd57 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:19.792062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d478b-60e9-48bb-8341-f243c49599dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:19.792704 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d478b-60e9-48bb-8341-f243c49599dc None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:19.792989 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d478b-60e9-48bb-8341-f243c49599dc None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:19.793259 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d478b-60e9-48bb-8341-f243c49599dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:19.793982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d478b-60e9-48bb-8341-f243c49599dc None admin] RBAC: Adding query filter params (role.id=c5946f310572439687baeed49acd4336, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=61ff29258a214817b8fc15b51fe8b3e6) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:19.794291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d478b-60e9-48bb-8341-f243c49599dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:19.795714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d478b-60e9-48bb-8341-f243c49599dc None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:19.804031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 489/979] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:14:19 2022] GET /identity/v3/role_assignments?user.id=61ff29258a214817b8fc15b51fe8b3e6&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=c5946f310572439687baeed49acd4336 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:20.435081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9f71b-2d2c-477d-a99c-202248034d72 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:20.435647 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9f71b-2d2c-477d-a99c-202248034d72 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:20.435647 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9f71b-2d2c-477d-a99c-202248034d72 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:20.437186 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 491/980] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:20.443185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efd2bb-8c7d-41f5-a61c-e8b641a711a0 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:20.443758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efd2bb-8c7d-41f5-a61c-e8b641a711a0 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:20.443758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efd2bb-8c7d-41f5-a61c-e8b641a711a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:20.463623 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-b3efd2bb-8c7d-41f5-a61c-e8b641a711a0 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:20.464985 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b3efd2bb-8c7d-41f5-a61c-e8b641a711a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:20.499981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:20.544860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 490/981] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:20.567641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9f71b-2d2c-477d-a99c-202248034d72 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:20.672125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af9f71b-2d2c-477d-a99c-202248034d72 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:20.706126 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb28261-96ff-40a3-b8ea-37117555bf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:20.706683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb28261-96ff-40a3-b8ea-37117555bf71 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:20.706961 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb28261-96ff-40a3-b8ea-37117555bf71 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:20.707230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb28261-96ff-40a3-b8ea-37117555bf71 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:20.707807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb28261-96ff-40a3-b8ea-37117555bf71 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:20.709210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb28261-96ff-40a3-b8ea-37117555bf71 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:20.718084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-9fb28261-96ff-40a3-b8ea-37117555bf71 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 19:14:20.718084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:20.718084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:20.718084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:20.718084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:20.718084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:20.718084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:20.718084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:20.718084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 19:14:20.718084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:20.718084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:20.718084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:20.718084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:20.718084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:20.718084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:20.718084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:20.718084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:20.719059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:20.719059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:20.719059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:20.719059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:20.719059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:20.719059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:20.719059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:20.719059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:20.719059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:20.719059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:20.719059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:20.719059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:20.719059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:20.719059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:20.719059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:20.719059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:20.720030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:20.720030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:20.720030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:20.720030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:20.720030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:20.720030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:20.720030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:20.720030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:20.720030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:20.720030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:20.720030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:20.720030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:20.720030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 19:14:20.720030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:20.720030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 492/982] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Aug 7 19:14:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:20.725501 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efd2bb-8c7d-41f5-a61c-e8b641a711a0 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:20.826767 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efd2bb-8c7d-41f5-a61c-e8b641a711a0 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:20.860615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe9532-6b4a-4179-a922-b339d7f7409c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:20.861231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe9532-6b4a-4179-a922-b339d7f7409c None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:20.861506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe9532-6b4a-4179-a922-b339d7f7409c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:20.861773 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fe9532-6b4a-4179-a922-b339d7f7409c None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:20.862399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe9532-6b4a-4179-a922-b339d7f7409c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:20.862701 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe9532-6b4a-4179-a922-b339d7f7409c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:20.864086 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fe9532-6b4a-4179-a922-b339d7f7409c None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:20.873421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 491/983] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Aug 7 19:14:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:20.874646 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb28261-96ff-40a3-b8ea-37117555bf71 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:20.926234 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:20.984134 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb28261-96ff-40a3-b8ea-37117555bf71 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:21.026482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a3178-6059-46f9-833f-57df13da6895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:21.027156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a3178-6059-46f9-833f-57df13da6895 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:21.027434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a3178-6059-46f9-833f-57df13da6895 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:21.027704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4a3178-6059-46f9-833f-57df13da6895 None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:21.028371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a3178-6059-46f9-833f-57df13da6895 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:21.028673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a3178-6059-46f9-833f-57df13da6895 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:21.030067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4a3178-6059-46f9-833f-57df13da6895 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:21.035424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 493/984] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Aug 7 19:14:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:21.041353 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe9532-6b4a-4179-a922-b339d7f7409c None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:21.072648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:21.145282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fe9532-6b4a-4179-a922-b339d7f7409c None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:21.178286 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1032fa-4f52-44eb-b0b0-921e72b031da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:21.178888 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1032fa-4f52-44eb-b0b0-921e72b031da None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:21.179163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1032fa-4f52-44eb-b0b0-921e72b031da None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:21.179431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1032fa-4f52-44eb-b0b0-921e72b031da None admin] PATH_INFO: `/v3/endpoints` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:21.180167 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1032fa-4f52-44eb-b0b0-921e72b031da None admin] RBAC: Adding query filter params (interface=admin, service_id=68b5208820074055b91e941db124b2d2, region_id=RegionOne) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:21.180473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1032fa-4f52-44eb-b0b0-921e72b031da None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:21.181844 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1032fa-4f52-44eb-b0b0-921e72b031da None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:21.187493 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 492/985] 10.222.0.17 () {56 vars in 1249 bytes} [Sun Aug 7 19:14:21 2022] GET /identity/v3/endpoints?service_id=68b5208820074055b91e941db124b2d2&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:21.193178 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a3178-6059-46f9-833f-57df13da6895 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:21.295980 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4a3178-6059-46f9-833f-57df13da6895 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:21.309731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:21.332245 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c1eec-eebd-4f40-b4e1-e377ac0aff18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:21.332866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c1eec-eebd-4f40-b4e1-e377ac0aff18 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:21.333141 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c1eec-eebd-4f40-b4e1-e377ac0aff18 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:21.333409 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c1eec-eebd-4f40-b4e1-e377ac0aff18 None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:21.334019 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c1eec-eebd-4f40-b4e1-e377ac0aff18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:21.335396 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c1eec-eebd-4f40-b4e1-e377ac0aff18 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:21.341670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 494/986] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Aug 7 19:14:21 2022] GET /identity/v3/services => generated 1497 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:14:21.349386 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c56e82-7272-45a5-a0ce-3690e9317ff1 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:21.349637 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c56e82-7272-45a5-a0ce-3690e9317ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:21.349926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c56e82-7272-45a5-a0ce-3690e9317ff1 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:21.351559 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 493/987] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:21.357435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e22b0-5207-40dd-bac6-6c52d1445690 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:21.357826 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e22b0-5207-40dd-bac6-6c52d1445690 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:21.357989 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e22b0-5207-40dd-bac6-6c52d1445690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:21.378631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-d83e22b0-5207-40dd-bac6-6c52d1445690 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:21.380286 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-d83e22b0-5207-40dd-bac6-6c52d1445690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:21.460116 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 495/988] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:21.483410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c56e82-7272-45a5-a0ce-3690e9317ff1 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:21.581057 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:21.591886 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c56e82-7272-45a5-a0ce-3690e9317ff1 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:21.628181 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d6d61f-ffda-4365-a959-cb0ca111688e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:21.628794 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6d61f-ffda-4365-a959-cb0ca111688e None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:21.629080 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6d61f-ffda-4365-a959-cb0ca111688e None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:21.629344 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d6d61f-ffda-4365-a959-cb0ca111688e None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:21.630004 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6d61f-ffda-4365-a959-cb0ca111688e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:21.631375 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d6d61f-ffda-4365-a959-cb0ca111688e None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:21.646459 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 494/989] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Aug 7 19:14:21 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:14:21.825480 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d4e88-ed57-438f-9478-d630cc56d4f6 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:21.825765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d4e88-ed57-438f-9478-d630cc56d4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:21.826036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d4e88-ed57-438f-9478-d630cc56d4f6 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:21.827566 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 496/990] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:21.833126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b52b4c-9a39-48ca-b078-8d33a62455c8 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:21.833418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b52b4c-9a39-48ca-b078-8d33a62455c8 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:21.833703 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b52b4c-9a39-48ca-b078-8d33a62455c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:21.852524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-f8b52b4c-9a39-48ca-b078-8d33a62455c8 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:21.854075 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f8b52b4c-9a39-48ca-b078-8d33a62455c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:21.932950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 495/991] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:21 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:21.953850 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d4e88-ed57-438f-9478-d630cc56d4f6 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:22.064438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d4e88-ed57-438f-9478-d630cc56d4f6 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:22.072754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:22.101806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d14f6-693f-4843-ae9a-cc0ecb6ff31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:22.102353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d14f6-693f-4843-ae9a-cc0ecb6ff31b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:22.102622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d14f6-693f-4843-ae9a-cc0ecb6ff31b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:22.102885 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d14f6-693f-4843-ae9a-cc0ecb6ff31b None admin] PATH_INFO: `/v3/services/image` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:22.103468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d14f6-693f-4843-ae9a-cc0ecb6ff31b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:22.104906 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d14f6-693f-4843-ae9a-cc0ecb6ff31b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:22.114157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-914d14f6-693f-4843-ae9a-cc0ecb6ff31b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 19:14:22.114157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:22.114157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:22.114157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:22.114157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:22.114157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:22.114157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:22.114157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:22.114157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 19:14:22.114157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:22.114157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:22.114157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:22.114157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:22.114157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:22.114157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:22.114157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:22.114157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:22.115189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:22.115189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:22.115189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:22.115189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:22.115189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:22.115189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:22.115189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:22.115189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:22.115189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:22.115189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:22.115189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:22.115189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:22.115189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:22.115189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:22.115189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:22.115189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:22.116085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:22.116085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:22.116085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:22.116085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:22.116085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:22.116085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:22.116085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:22.116085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:22.116085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:22.116085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:22.116085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:22.116085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:22.116085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 19:14:22.116085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:22.117636 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 497/992] 10.222.0.17 () {56 vars in 1105 bytes} [Sun Aug 7 19:14:21 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:22.121411 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b52b4c-9a39-48ca-b078-8d33a62455c8 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:22.202878 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:22.227975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b52b4c-9a39-48ca-b078-8d33a62455c8 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:22.262062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0783f-03ba-4e7a-91dc-06624f19a6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:22.262678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0783f-03ba-4e7a-91dc-06624f19a6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:22.262957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0783f-03ba-4e7a-91dc-06624f19a6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:22.263229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0783f-03ba-4e7a-91dc-06624f19a6e0 None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:22.263859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0783f-03ba-4e7a-91dc-06624f19a6e0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:22.264193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0783f-03ba-4e7a-91dc-06624f19a6e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:22.265651 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0783f-03ba-4e7a-91dc-06624f19a6e0 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:22.275221 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 496/993] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:22.276297 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d14f6-693f-4843-ae9a-cc0ecb6ff31b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:22.381998 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d14f6-693f-4843-ae9a-cc0ecb6ff31b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:22.421051 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972d6e1-73ee-4f1b-9068-90913e951c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:22.421680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972d6e1-73ee-4f1b-9068-90913e951c4a None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:22.421960 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972d6e1-73ee-4f1b-9068-90913e951c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:22.422268 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972d6e1-73ee-4f1b-9068-90913e951c4a None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:22.422874 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972d6e1-73ee-4f1b-9068-90913e951c4a None admin] RBAC: Adding query filter params (type=image) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:22.423206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972d6e1-73ee-4f1b-9068-90913e951c4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:22.424616 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972d6e1-73ee-4f1b-9068-90913e951c4a None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:22.429808 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 498/994] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:22 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:23.374057 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424998e4-b133-4fed-80a8-29984c53e22a None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:23.374581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424998e4-b133-4fed-80a8-29984c53e22a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:23.374581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424998e4-b133-4fed-80a8-29984c53e22a None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:23.376138 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 497/995] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:23.381335 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30654bc1-2101-4f82-8217-671d1b4946b2 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:23.381766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30654bc1-2101-4f82-8217-671d1b4946b2 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:23.381843 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30654bc1-2101-4f82-8217-671d1b4946b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:23.401338 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-30654bc1-2101-4f82-8217-671d1b4946b2 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:23.402883 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-30654bc1-2101-4f82-8217-671d1b4946b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:23.433011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:23.481771 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 499/996] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:23 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:23.506536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424998e4-b133-4fed-80a8-29984c53e22a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:23.616226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424998e4-b133-4fed-80a8-29984c53e22a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:23.651127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be188d2-53c0-4d67-a5e7-d81b4ff4dbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:23.651751 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be188d2-53c0-4d67-a5e7-d81b4ff4dbd4 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:23.652050 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be188d2-53c0-4d67-a5e7-d81b4ff4dbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:23.652337 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be188d2-53c0-4d67-a5e7-d81b4ff4dbd4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:23.652927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be188d2-53c0-4d67-a5e7-d81b4ff4dbd4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:23.654300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be188d2-53c0-4d67-a5e7-d81b4ff4dbd4 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:23.663127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-7be188d2-53c0-4d67-a5e7-d81b4ff4dbd4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 19:14:23.663127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:23.663127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:23.663127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:23.663127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:23.663127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:23.663127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:23.663127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:23.663127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 19:14:23.663127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:23.663127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:23.663127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:23.663127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:23.663127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:23.663127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:23.663127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:23.663127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:23.664242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:23.664242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:23.664242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:23.664242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:23.664242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:23.664242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:23.664242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:23.664242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:23.664242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:23.664242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:23.664242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:23.664242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:23.664242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:23.664242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:23.664242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:23.664242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:23.670454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:23.670454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:23.670454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:23.670454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:23.670454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:23.670454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:23.670454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:23.670454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:23.670454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:23.670454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:23.670454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:23.670454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:23.670454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 19:14:23.670454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:23.670454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 498/997] 10.222.0.17 () {56 vars in 1126 bytes} [Sun Aug 7 19:14:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:23.671257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be188d2-53c0-4d67-a5e7-d81b4ff4dbd4 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:23.671325 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61b7a3-61eb-40b1-b161-f0f714ec2f72 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:23.671325 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61b7a3-61eb-40b1-b161-f0f714ec2f72 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:23.671325 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61b7a3-61eb-40b1-b161-f0f714ec2f72 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:23.672495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 500/998] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:23.677401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eac17d-6a7a-43b3-a1eb-380ead7d6d37 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:23.677684 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eac17d-6a7a-43b3-a1eb-380ead7d6d37 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:23.677953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eac17d-6a7a-43b3-a1eb-380ead7d6d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:23.680896 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:23.697197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-66eac17d-6a7a-43b3-a1eb-380ead7d6d37 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:23.698585 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-66eac17d-6a7a-43b3-a1eb-380ead7d6d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:23.772161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 501/999] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:23 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 19:14:23.780365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be188d2-53c0-4d67-a5e7-d81b4ff4dbd4 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:23.791545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eac17d-6a7a-43b3-a1eb-380ead7d6d37 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:23.817502 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46917fb0-4162-449f-954b-87531683b29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:23.818593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46917fb0-4162-449f-954b-87531683b29e None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:23.819121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46917fb0-4162-449f-954b-87531683b29e None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:23.819580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46917fb0-4162-449f-954b-87531683b29e None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:23.820450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46917fb0-4162-449f-954b-87531683b29e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:23.821032 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46917fb0-4162-449f-954b-87531683b29e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:23.822712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46917fb0-4162-449f-954b-87531683b29e None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:23.832717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 499/1000] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Aug 7 19:14:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:23.838351 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46917fb0-4162-449f-954b-87531683b29e None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:23.892014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:23.900334 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eac17d-6a7a-43b3-a1eb-380ead7d6d37 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:23.908111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:23.935620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652f40b-3cd7-4e25-9d31-a3f7b3dff509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:23.936306 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652f40b-3cd7-4e25-9d31-a3f7b3dff509 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:23.936583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652f40b-3cd7-4e25-9d31-a3f7b3dff509 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:23.936861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652f40b-3cd7-4e25-9d31-a3f7b3dff509 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:23.937450 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652f40b-3cd7-4e25-9d31-a3f7b3dff509 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:23.938834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652f40b-3cd7-4e25-9d31-a3f7b3dff509 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:23.942912 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46917fb0-4162-449f-954b-87531683b29e None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:23.947754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-2652f40b-3cd7-4e25-9d31-a3f7b3dff509 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 19:14:23.947754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:23.947754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:23.947754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:23.947754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:23.947754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:23.947754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:23.947754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:23.947754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 19:14:23.947754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:23.947754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:23.947754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:23.947754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:23.947754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:23.947754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:23.947754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:23.947754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:23.949025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:23.949025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:23.949025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:23.949025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:23.949025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:23.949025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:23.949025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:23.949025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:23.949025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:23.949025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:23.949025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:23.949025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:23.949025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:23.949025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:23.949025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:23.949025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:23.950061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:23.950061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:23.950061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:23.950061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:23.950061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:23.950061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:23.950061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:23.950061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:23.950061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:23.950061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:23.950061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:23.950061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:23.950061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 19:14:23.950061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:23.954086 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 502/1001] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:14:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:23.954953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652f40b-3cd7-4e25-9d31-a3f7b3dff509 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:23.982677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7fd68-bc33-403d-9a5b-9567d0a04368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:23.983298 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7fd68-bc33-403d-9a5b-9567d0a04368 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:23.983580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7fd68-bc33-403d-9a5b-9567d0a04368 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:23.983853 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7fd68-bc33-403d-9a5b-9567d0a04368 None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:23.984522 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7fd68-bc33-403d-9a5b-9567d0a04368 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:23.984829 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7fd68-bc33-403d-9a5b-9567d0a04368 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:23.986206 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7fd68-bc33-403d-9a5b-9567d0a04368 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:23.991074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 500/1002] 10.222.0.17 () {56 vars in 1122 bytes} [Sun Aug 7 19:14:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:23.997207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7fd68-bc33-403d-9a5b-9567d0a04368 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:24.064641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652f40b-3cd7-4e25-9d31-a3f7b3dff509 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:24.102925 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f35673-865d-4e71-9434-27c7832b5de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:24.103563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f35673-865d-4e71-9434-27c7832b5de9 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:24.103844 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f35673-865d-4e71-9434-27c7832b5de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:24.104197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f35673-865d-4e71-9434-27c7832b5de9 None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:24.104864 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f35673-865d-4e71-9434-27c7832b5de9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:24.105361 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f35673-865d-4e71-9434-27c7832b5de9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:24.106646 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f35673-865d-4e71-9434-27c7832b5de9 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:24.114221 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7fd68-bc33-403d-9a5b-9567d0a04368 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:24.117538 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 503/1003] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Aug 7 19:14:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:24.117810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f35673-865d-4e71-9434-27c7832b5de9 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:24.144509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:24.150901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa919006-20ac-442f-a3da-6326eb077bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:24.151588 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa919006-20ac-442f-a3da-6326eb077bb4 None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:24.151854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa919006-20ac-442f-a3da-6326eb077bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:24.152197 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa919006-20ac-442f-a3da-6326eb077bb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:24.152844 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa919006-20ac-442f-a3da-6326eb077bb4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:24.154216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa919006-20ac-442f-a3da-6326eb077bb4 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:24.159612 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:24.174083 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 501/1004] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:14:23 2022] POST /identity/v3/endpoints => generated 325 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:14:24.252853 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f35673-865d-4e71-9434-27c7832b5de9 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:24.298898 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdbb261-b3cd-4d2d-b4a8-df2fd272af49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:24.299909 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbb261-b3cd-4d2d-b4a8-df2fd272af49 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:24.300413 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbb261-b3cd-4d2d-b4a8-df2fd272af49 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:24.300858 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbb261-b3cd-4d2d-b4a8-df2fd272af49 None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:24.301688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdbb261-b3cd-4d2d-b4a8-df2fd272af49 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:24.302188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdbb261-b3cd-4d2d-b4a8-df2fd272af49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:24.303894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdbb261-b3cd-4d2d-b4a8-df2fd272af49 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:24.310754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 504/1005] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Aug 7 19:14:24 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:24.317039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa919006-20ac-442f-a3da-6326eb077bb4 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:24.422696 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa919006-20ac-442f-a3da-6326eb077bb4 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:24.445909 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4da46d-3f19-42b2-92ee-de67e4215056 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:24.446442 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4da46d-3f19-42b2-92ee-de67e4215056 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:24.446672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4da46d-3f19-42b2-92ee-de67e4215056 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:24.448167 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 505/1006] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:24.454074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd151a69-46c5-44ef-b5be-2ef32f0229d3 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:24.454360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd151a69-46c5-44ef-b5be-2ef32f0229d3 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:24.454635 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd151a69-46c5-44ef-b5be-2ef32f0229d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:24.457267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738f6b8-2816-4515-bb21-a64b2b14a59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:24.457909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738f6b8-2816-4515-bb21-a64b2b14a59d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:24.458219 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738f6b8-2816-4515-bb21-a64b2b14a59d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:24.458449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738f6b8-2816-4515-bb21-a64b2b14a59d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:24.459148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738f6b8-2816-4515-bb21-a64b2b14a59d None admin] RBAC: Adding query filter params (interface=public, service_id=bb8bd6a887dc44478634a3c73f8fa3b0, region_id=RegionOne) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:24.459453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738f6b8-2816-4515-bb21-a64b2b14a59d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:24.460849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738f6b8-2816-4515-bb21-a64b2b14a59d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:24.466464 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 502/1007] 10.222.0.17 () {56 vars in 1251 bytes} [Sun Aug 7 19:14:24 2022] GET /identity/v3/endpoints?service_id=bb8bd6a887dc44478634a3c73f8fa3b0&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:24.471769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738f6b8-2816-4515-bb21-a64b2b14a59d None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:24.478965 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-fd151a69-46c5-44ef-b5be-2ef32f0229d3 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:24.480317 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-fd151a69-46c5-44ef-b5be-2ef32f0229d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:24.535689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:24.571493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 506/1008] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:24 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 19:14:24.585781 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738f6b8-2816-4515-bb21-a64b2b14a59d None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:24.586455 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd151a69-46c5-44ef-b5be-2ef32f0229d3 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:24.601723 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:24.625093 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5cf32-e1f1-4520-8d81-0810b6483ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:24.625740 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5cf32-e1f1-4520-8d81-0810b6483ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:24.626017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5cf32-e1f1-4520-8d81-0810b6483ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:24.626292 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5cf32-e1f1-4520-8d81-0810b6483ff8 None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:24.626921 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5cf32-e1f1-4520-8d81-0810b6483ff8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:24.628345 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5cf32-e1f1-4520-8d81-0810b6483ff8 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:24.635456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 503/1009] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Aug 7 19:14:24 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:14:24.706397 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd151a69-46c5-44ef-b5be-2ef32f0229d3 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:24.746027 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ba5e5-7f52-4bc3-94db-e641302ddc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:24.746663 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ba5e5-7f52-4bc3-94db-e641302ddc1f None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:24.746933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ba5e5-7f52-4bc3-94db-e641302ddc1f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:24.747200 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ba5e5-7f52-4bc3-94db-e641302ddc1f None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:24.747853 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ba5e5-7f52-4bc3-94db-e641302ddc1f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:24.749294 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ba5e5-7f52-4bc3-94db-e641302ddc1f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:24.776026 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 507/1010] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Aug 7 19:14:24 2022] POST /identity/v3/services => generated 242 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:14:26.216243 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb0a5f-41a5-44d9-8d06-88b81061e729 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:26.216788 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb0a5f-41a5-44d9-8d06-88b81061e729 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:26.216788 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb0a5f-41a5-44d9-8d06-88b81061e729 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:26.218098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 504/1011] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:26.223667 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd775f5-c3c2-431c-a3d5-247d7440451d None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:26.224536 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd775f5-c3c2-431c-a3d5-247d7440451d None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:26.224791 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd775f5-c3c2-431c-a3d5-247d7440451d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:26.243723 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-1dd775f5-c3c2-431c-a3d5-247d7440451d None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:26.244976 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-1dd775f5-c3c2-431c-a3d5-247d7440451d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:26.314671 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:26.332299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 508/1012] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:26 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:26.348579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb0a5f-41a5-44d9-8d06-88b81061e729 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:26.453406 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb0a5f-41a5-44d9-8d06-88b81061e729 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:26.477280 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:26.489876 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860b8bb-ad22-4abb-b08b-8a0b3f397f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:26.490512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860b8bb-ad22-4abb-b08b-8a0b3f397f7a None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:26.490785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860b8bb-ad22-4abb-b08b-8a0b3f397f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:26.491052 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860b8bb-ad22-4abb-b08b-8a0b3f397f7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:26.492216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860b8bb-ad22-4abb-b08b-8a0b3f397f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:26.493651 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860b8bb-ad22-4abb-b08b-8a0b3f397f7a None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:26.495468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 505/1013] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:26.501153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd775f5-c3c2-431c-a3d5-247d7440451d None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:26.600711 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd775f5-c3c2-431c-a3d5-247d7440451d None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:26.635011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724b5ae-d60c-435c-b4f6-271a3e2a7e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:26.635631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724b5ae-d60c-435c-b4f6-271a3e2a7e26 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:26.635941 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724b5ae-d60c-435c-b4f6-271a3e2a7e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:26.636250 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724b5ae-d60c-435c-b4f6-271a3e2a7e26 None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:26.636902 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724b5ae-d60c-435c-b4f6-271a3e2a7e26 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:26.638348 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724b5ae-d60c-435c-b4f6-271a3e2a7e26 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:26.672033 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 509/1014] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Aug 7 19:14:26 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:14:26.714225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a340fc-52d2-4c65-af23-461b806112af None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:26.714505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a340fc-52d2-4c65-af23-461b806112af None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:26.714780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a340fc-52d2-4c65-af23-461b806112af None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:26.716225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 506/1015] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:26.722211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0931301-dac7-4a22-a5f4-94d4e8537280 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:26.722582 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0931301-dac7-4a22-a5f4-94d4e8537280 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:26.722815 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0931301-dac7-4a22-a5f4-94d4e8537280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:26.745823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-b0931301-dac7-4a22-a5f4-94d4e8537280 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:26.750547 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b0931301-dac7-4a22-a5f4-94d4e8537280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:26.773295 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb22f0-8089-423c-9395-96a58276bd08 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:26.773617 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb22f0-8089-423c-9395-96a58276bd08 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:26.773850 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb22f0-8089-423c-9395-96a58276bd08 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:26.775350 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 507/1016] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:26.780623 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db981d7-02a6-4459-ac30-bf70e994e4dc None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:26.780903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db981d7-02a6-4459-ac30-bf70e994e4dc None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:26.781145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db981d7-02a6-4459-ac30-bf70e994e4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:26.801891 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-2db981d7-02a6-4459-ac30-bf70e994e4dc None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:26.803476 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-2db981d7-02a6-4459-ac30-bf70e994e4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:26.806029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:26.831520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 510/1017] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:26 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:26.854280 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0931301-dac7-4a22-a5f4-94d4e8537280 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:26.883711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 508/1018] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:26 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 19:14:26.905754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db981d7-02a6-4459-ac30-bf70e994e4dc None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:26.978906 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0931301-dac7-4a22-a5f4-94d4e8537280 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:27.023439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205d35b-f58b-48a1-a552-a00a8d507881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:27.024620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205d35b-f58b-48a1-a552-a00a8d507881 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:27.025043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205d35b-f58b-48a1-a552-a00a8d507881 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:27.025311 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205d35b-f58b-48a1-a552-a00a8d507881 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:27.025911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205d35b-f58b-48a1-a552-a00a8d507881 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:27.027167 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db981d7-02a6-4459-ac30-bf70e994e4dc None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:27.027440 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205d35b-f58b-48a1-a552-a00a8d507881 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:27.037044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-8205d35b-f58b-48a1-a552-a00a8d507881 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 19:14:27.037044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:27.037044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:27.037044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:27.037044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:27.037044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:27.037044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:27.037044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:27.037044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 19:14:27.037044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:27.037044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:27.037044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:27.037044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:27.037044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:27.037044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:27.037044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:27.037044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:27.038050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:27.038050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:27.038050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:27.038050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:27.038050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:27.038050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:27.038050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:27.038050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:27.038050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:27.038050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:27.038050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:27.038050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:27.038050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:27.038050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:27.038050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:27.038050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:27.038873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:27.038873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:27.038873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:27.038873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:27.038873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:27.038873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:27.038873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:27.038873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:27.038873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:27.038873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:27.038873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:27.038873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:27.038873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 19:14:27.038873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:27.043626 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 511/1019] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:14:26 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:27.044388 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205d35b-f58b-48a1-a552-a00a8d507881 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:27.052183 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:27.069469 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4591c-7020-4652-b562-53d19529edc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:27.070235 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4591c-7020-4652-b562-53d19529edc7 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:27.070512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4591c-7020-4652-b562-53d19529edc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:27.070945 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4591c-7020-4652-b562-53d19529edc7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:27.071861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4591c-7020-4652-b562-53d19529edc7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:27.073553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4591c-7020-4652-b562-53d19529edc7 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:27.083592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-7ab4591c-7020-4652-b562-53d19529edc7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 19:14:27.083592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:27.083592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:27.083592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:27.083592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:27.083592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:27.083592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:27.083592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:27.083592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 19:14:27.083592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:27.083592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:27.083592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:27.083592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:27.083592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:27.083592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:27.083592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:27.083592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:27.084671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:27.084671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:27.084671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:27.084671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:27.084671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:27.084671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:27.084671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:27.084671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:27.084671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:27.084671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:27.084671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:27.084671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:27.084671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:27.084671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:27.084671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:27.084671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:27.085509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:27.085509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:27.085509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:27.085509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:27.085509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:27.085509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:27.085509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:27.085509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:27.085509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:27.085509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:27.085509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:27.085509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:27.085509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 19:14:27.085509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:27.085509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 509/1020] 10.222.0.17 () {56 vars in 1105 bytes} [Sun Aug 7 19:14:26 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:27.090841 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4591c-7020-4652-b562-53d19529edc7 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:27.097238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:27.158895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205d35b-f58b-48a1-a552-a00a8d507881 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:27.194945 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4591c-7020-4652-b562-53d19529edc7 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:27.197947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe46b3c-7acc-4c0f-b01d-c0207a3619e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:27.198605 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe46b3c-7acc-4c0f-b01d-c0207a3619e7 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:27.198879 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe46b3c-7acc-4c0f-b01d-c0207a3619e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:27.199145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe46b3c-7acc-4c0f-b01d-c0207a3619e7 None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:27.200675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe46b3c-7acc-4c0f-b01d-c0207a3619e7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:27.200981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe46b3c-7acc-4c0f-b01d-c0207a3619e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:27.202375 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe46b3c-7acc-4c0f-b01d-c0207a3619e7 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:27.213098 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 512/1021] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Aug 7 19:14:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:27.214323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe46b3c-7acc-4c0f-b01d-c0207a3619e7 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:27.237511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8523e6d-564a-4782-836a-752b924b596a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:27.238334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8523e6d-564a-4782-836a-752b924b596a None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:27.238457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8523e6d-564a-4782-836a-752b924b596a None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:27.238660 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8523e6d-564a-4782-836a-752b924b596a None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:27.239297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8523e6d-564a-4782-836a-752b924b596a None admin] RBAC: Adding query filter params (name=image) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:27.239601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8523e6d-564a-4782-836a-752b924b596a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:27.241044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8523e6d-564a-4782-836a-752b924b596a None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:27.248031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 510/1022] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:27.252193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8523e6d-564a-4782-836a-752b924b596a None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:27.311942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:27.329273 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe46b3c-7acc-4c0f-b01d-c0207a3619e7 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:27.339499 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:27.367210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b31643-0694-403f-8f8a-5843f9b0a8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:27.367842 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b31643-0694-403f-8f8a-5843f9b0a8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:27.368153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b31643-0694-403f-8f8a-5843f9b0a8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:27.368426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b31643-0694-403f-8f8a-5843f9b0a8e3 None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:27.369057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b31643-0694-403f-8f8a-5843f9b0a8e3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:27.369353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b31643-0694-403f-8f8a-5843f9b0a8e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:27.370847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b31643-0694-403f-8f8a-5843f9b0a8e3 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:27.375784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 513/1023] 10.222.0.17 () {56 vars in 1112 bytes} [Sun Aug 7 19:14:27 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:27.381498 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b31643-0694-403f-8f8a-5843f9b0a8e3 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:27.401541 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8523e6d-564a-4782-836a-752b924b596a None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:27.450039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b4635-7ffb-4cd5-9760-280b3ff37340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:27.450649 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b4635-7ffb-4cd5-9760-280b3ff37340 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:27.450926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b4635-7ffb-4cd5-9760-280b3ff37340 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:27.451194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b4635-7ffb-4cd5-9760-280b3ff37340 None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:27.451824 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b4635-7ffb-4cd5-9760-280b3ff37340 None admin] RBAC: Adding query filter params (type=image) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:27.452158 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b4635-7ffb-4cd5-9760-280b3ff37340 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:27.453516 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b4635-7ffb-4cd5-9760-280b3ff37340 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:27.459143 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 511/1024] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:27.477629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b4635-7ffb-4cd5-9760-280b3ff37340 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:27.507753 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b31643-0694-403f-8f8a-5843f9b0a8e3 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:27.544565 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34493c14-8eec-4922-8afb-a1f120bafdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:27.545177 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34493c14-8eec-4922-8afb-a1f120bafdd5 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:27.545451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34493c14-8eec-4922-8afb-a1f120bafdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:27.545722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34493c14-8eec-4922-8afb-a1f120bafdd5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:27.546379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34493c14-8eec-4922-8afb-a1f120bafdd5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:27.547753 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34493c14-8eec-4922-8afb-a1f120bafdd5 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:27.580030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 514/1025] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:14:27 2022] POST /identity/v3/endpoints => generated 334 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:14:27.594962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b4635-7ffb-4cd5-9760-280b3ff37340 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:27.610475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:27.640919 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa12dd2-2d8b-4860-9385-21ac14a6184e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:27.641560 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa12dd2-2d8b-4860-9385-21ac14a6184e None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:27.641837 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa12dd2-2d8b-4860-9385-21ac14a6184e None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:27.642104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa12dd2-2d8b-4860-9385-21ac14a6184e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:27.642813 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa12dd2-2d8b-4860-9385-21ac14a6184e None admin] RBAC: Adding query filter params (interface=public, service_id=02ff3ac666b1418bb723347aedfae2c7, region_id=RegionOne) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:27.643194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa12dd2-2d8b-4860-9385-21ac14a6184e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:27.644761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa12dd2-2d8b-4860-9385-21ac14a6184e None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:27.651154 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 512/1026] 10.222.0.17 () {56 vars in 1251 bytes} [Sun Aug 7 19:14:27 2022] GET /identity/v3/endpoints?service_id=02ff3ac666b1418bb723347aedfae2c7&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:27.657009 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34493c14-8eec-4922-8afb-a1f120bafdd5 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:27.689951 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:27.772417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34493c14-8eec-4922-8afb-a1f120bafdd5 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:27.810269 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0d92a-0c78-4baa-888d-decaea480c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:27.810901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0d92a-0c78-4baa-888d-decaea480c47 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:27.811185 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0d92a-0c78-4baa-888d-decaea480c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:27.811452 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0d92a-0c78-4baa-888d-decaea480c47 None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:27.812160 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0d92a-0c78-4baa-888d-decaea480c47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:27.813558 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0d92a-0c78-4baa-888d-decaea480c47 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:27.821451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 515/1027] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Aug 7 19:14:27 2022] GET /identity/v3/services => generated 1957 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:14:28.786095 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552901f5-e763-409f-9a07-31528067f556 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:28.786683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552901f5-e763-409f-9a07-31528067f556 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:28.786683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552901f5-e763-409f-9a07-31528067f556 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:28.788202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 513/1028] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:28.793539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d0185-fc01-4251-9897-2270e3cf7a3f None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:28.794039 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d0185-fc01-4251-9897-2270e3cf7a3f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:28.794149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d0185-fc01-4251-9897-2270e3cf7a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:28.814706 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-e09d0185-fc01-4251-9897-2270e3cf7a3f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:28.815909 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e09d0185-fc01-4251-9897-2270e3cf7a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:28.888355 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 516/1029] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:28.912409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552901f5-e763-409f-9a07-31528067f556 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:29.026200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552901f5-e763-409f-9a07-31528067f556 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:29.060595 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d44408-e0e3-4e1e-87a8-9c1fd9311e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:29.061142 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d44408-e0e3-4e1e-87a8-9c1fd9311e04 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:29.061414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d44408-e0e3-4e1e-87a8-9c1fd9311e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:29.061718 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d44408-e0e3-4e1e-87a8-9c1fd9311e04 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:29.062854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d44408-e0e3-4e1e-87a8-9c1fd9311e04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:29.064303 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d44408-e0e3-4e1e-87a8-9c1fd9311e04 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:29.068215 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 514/1030] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:29.072384 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d0185-fc01-4251-9897-2270e3cf7a3f None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:29.097626 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:29.177018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d0185-fc01-4251-9897-2270e3cf7a3f None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:29.212551 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bdee82-66e2-43c0-88d3-4c13b2f2c009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:29.213577 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdee82-66e2-43c0-88d3-4c13b2f2c009 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:29.214098 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdee82-66e2-43c0-88d3-4c13b2f2c009 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:29.214586 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdee82-66e2-43c0-88d3-4c13b2f2c009 None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:29.215448 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bdee82-66e2-43c0-88d3-4c13b2f2c009 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:29.217144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bdee82-66e2-43c0-88d3-4c13b2f2c009 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:29.242085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 517/1031] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:14:29 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:14:29.701118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c722396-7ecd-4412-8849-e56731375bac None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:29.701535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c722396-7ecd-4412-8849-e56731375bac None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:29.701671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c722396-7ecd-4412-8849-e56731375bac None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:29.703195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 515/1032] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:29.708668 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f6cf9-d5aa-4b6c-a253-fabeeb3d45e5 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:29.708954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f6cf9-d5aa-4b6c-a253-fabeeb3d45e5 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:29.709225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672f6cf9-d5aa-4b6c-a253-fabeeb3d45e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:29.728838 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-672f6cf9-d5aa-4b6c-a253-fabeeb3d45e5 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:29.730174 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-672f6cf9-d5aa-4b6c-a253-fabeeb3d45e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:29.804027 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 518/1033] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:29.825825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c722396-7ecd-4412-8849-e56731375bac None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:29.841385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b55a8-b7e4-4c8b-9abe-252d0eb74bae None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:29.841666 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b55a8-b7e4-4c8b-9abe-252d0eb74bae None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:29.841935 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b55a8-b7e4-4c8b-9abe-252d0eb74bae None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:29.843292 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 519/1034] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:14:29.849075 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5c504-0544-43d0-b5cd-6da06341a87c None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:29.849358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5c504-0544-43d0-b5cd-6da06341a87c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:29.849632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5c504-0544-43d0-b5cd-6da06341a87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:29.868743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-66d5c504-0544-43d0-b5cd-6da06341a87c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:29.869959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-66d5c504-0544-43d0-b5cd-6da06341a87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:29.872057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:29.874764 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:29.938119 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c722396-7ecd-4412-8849-e56731375bac None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:29.949411 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 520/1035] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:29.971965 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5c504-0544-43d0-b5cd-6da06341a87c None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:29.977278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec601276-c272-42c0-8cd9-6ce53f246d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:29.977927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec601276-c272-42c0-8cd9-6ce53f246d3b None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:29.978207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec601276-c272-42c0-8cd9-6ce53f246d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:29.978483 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec601276-c272-42c0-8cd9-6ce53f246d3b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:29.989730 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec601276-c272-42c0-8cd9-6ce53f246d3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:14:29.994440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-ec601276-c272-42c0-8cd9-6ce53f246d3b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 19:14:29.994440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:29.994440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:29.994440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:29.994440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:29.994440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:29.994440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:29.994440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:29.994440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:29.994440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:29.994440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:29.994440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:29.994440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:14:29.994440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:14:29.994440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:14:29.994440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:14:29.995515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:14:29.995515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:14:29.995515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:14:29.995515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:14:29.995515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:29.995515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:29.995515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:29.995515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:29.995515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:29.995515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:29.995515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:29.995515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:29.995515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:29.995515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:29.995515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:29.995515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:29.996417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:29.996417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:29.996417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:29.996417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:29.996417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:14:29.996417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:14:29.996417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:14:29.996417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:14:29.996417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:14:29.996417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:14:29.996417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 19:14:29.996417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:29.996417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 516/1036] 10.222.0.17 () {56 vars in 1120 bytes} [Sun Aug 7 19:14:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:30.001680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec601276-c272-42c0-8cd9-6ce53f246d3b None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:30.090545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5c504-0544-43d0-b5cd-6da06341a87c None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:30.105940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:30.109430 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec601276-c272-42c0-8cd9-6ce53f246d3b None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:30.126028 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fba72a-c74c-48c1-91ee-5b175eb5d73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:30.126725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fba72a-c74c-48c1-91ee-5b175eb5d73c None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:30.126941 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fba72a-c74c-48c1-91ee-5b175eb5d73c None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:30.127206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fba72a-c74c-48c1-91ee-5b175eb5d73c None admin] PATH_INFO: `/v3/services/image` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:30.127793 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fba72a-c74c-48c1-91ee-5b175eb5d73c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:30.129189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fba72a-c74c-48c1-91ee-5b175eb5d73c None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:30.139012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-d9fba72a-c74c-48c1-91ee-5b175eb5d73c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 19:14:30.139012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:30.139012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:14:30.139012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:14:30.139012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:14:30.139012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:14:30.139012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:14:30.139012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:30.139012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 19:14:30.139012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:30.139012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:14:30.139012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:30.139012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:30.139012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:30.139012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:30.139012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:30.139012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:30.140036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:30.140036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:30.140036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:30.140036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:30.140036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:30.140036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:30.140036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:14:30.140036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:14:30.140036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:14:30.140036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:14:30.140036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:30.140036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:30.140036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:30.140036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:30.140036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:30.140036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:30.140911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:30.140911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:30.140911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:30.140911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:30.140911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:30.140911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:30.140911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:30.140911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:30.140911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:30.140911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:30.140911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:14:30.140911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:14:30.140911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 19:14:30.140911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:30.140911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 521/1037] 10.222.0.17 () {56 vars in 1105 bytes} [Sun Aug 7 19:14:29 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:30.145999 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fba72a-c74c-48c1-91ee-5b175eb5d73c None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:30.149664 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:30.158377 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f61a6f-1773-4d01-8a43-6e09c31f5e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:30.159292 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f61a6f-1773-4d01-8a43-6e09c31f5e49 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:30.159573 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f61a6f-1773-4d01-8a43-6e09c31f5e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:30.160001 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f61a6f-1773-4d01-8a43-6e09c31f5e49 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:30.160679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f61a6f-1773-4d01-8a43-6e09c31f5e49 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:30.161009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f61a6f-1773-4d01-8a43-6e09c31f5e49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:30.162453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f61a6f-1773-4d01-8a43-6e09c31f5e49 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:30.171101 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 517/1038] 10.222.0.17 () {56 vars in 1115 bytes} [Sun Aug 7 19:14:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:30.176862 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f61a6f-1773-4d01-8a43-6e09c31f5e49 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:30.249318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fba72a-c74c-48c1-91ee-5b175eb5d73c None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:30.278520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f61a6f-1773-4d01-8a43-6e09c31f5e49 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:30.284233 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee083799-9a1d-46cc-96f2-790c94903a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:30.284850 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee083799-9a1d-46cc-96f2-790c94903a80 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:30.285128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee083799-9a1d-46cc-96f2-790c94903a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:30.285401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee083799-9a1d-46cc-96f2-790c94903a80 None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:30.286152 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee083799-9a1d-46cc-96f2-790c94903a80 None admin] RBAC: Adding query filter params (name=image) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:30.286455 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee083799-9a1d-46cc-96f2-790c94903a80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:30.287977 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee083799-9a1d-46cc-96f2-790c94903a80 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:30.297999 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 522/1039] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:30.298867 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee083799-9a1d-46cc-96f2-790c94903a80 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:30.314671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d111b-c444-48e7-a1ce-66e9b194e052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:30.315463 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d111b-c444-48e7-a1ce-66e9b194e052 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:30.315749 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d111b-c444-48e7-a1ce-66e9b194e052 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:30.316094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d111b-c444-48e7-a1ce-66e9b194e052 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:30.328096 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d111b-c444-48e7-a1ce-66e9b194e052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:30.329571 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d111b-c444-48e7-a1ce-66e9b194e052 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:30.344801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-b16d111b-c444-48e7-a1ce-66e9b194e052 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:30.344801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:30.344801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:30.344801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:30.344801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:30.344801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:30.344801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:30.344801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:30.344801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:30.344801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:30.344801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:30.344801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:30.344801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:30.344801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:30.344801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:30.344801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:30.345801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:30.345801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:30.345801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:30.345801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:30.345801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:30.345801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:30.345801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:30.345801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:30.345801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:30.345801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:30.345801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:30.345801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:30.345801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:30.345801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:30.345801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:30.345801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:30.345801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:30.346934 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:30.346934 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:30.346934 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:30.346934 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 518/1040] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:30.351939 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d111b-c444-48e7-a1ce-66e9b194e052 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:30.368209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:30.368470 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:30.408294 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee083799-9a1d-46cc-96f2-790c94903a80 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:30.448559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cbf49-0647-47a4-8399-3270f3c98e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:30.449181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cbf49-0647-47a4-8399-3270f3c98e41 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:30.449456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cbf49-0647-47a4-8399-3270f3c98e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:30.449728 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cbf49-0647-47a4-8399-3270f3c98e41 None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:30.450361 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8cbf49-0647-47a4-8399-3270f3c98e41 None admin] RBAC: Adding query filter params (type=image) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:30.450659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8cbf49-0647-47a4-8399-3270f3c98e41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:30.452078 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8cbf49-0647-47a4-8399-3270f3c98e41 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:30.461760 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 523/1041] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:30.462700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cbf49-0647-47a4-8399-3270f3c98e41 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:30.463077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d111b-c444-48e7-a1ce-66e9b194e052 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:30.499671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af9906-336a-4d1a-8586-63200f3c49a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:30.500226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af9906-336a-4d1a-8586-63200f3c49a2 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:30.500500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af9906-336a-4d1a-8586-63200f3c49a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:30.500763 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af9906-336a-4d1a-8586-63200f3c49a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:30.501406 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af9906-336a-4d1a-8586-63200f3c49a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:30.501707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af9906-336a-4d1a-8586-63200f3c49a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:30.503066 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af9906-336a-4d1a-8586-63200f3c49a2 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:30.516339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 519/1042] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:30.522127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af9906-336a-4d1a-8586-63200f3c49a2 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:30.568871 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cbf49-0647-47a4-8399-3270f3c98e41 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:30.598161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:30.599556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:30.607439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44198412-ad08-40b5-aacd-6c4ab79ecb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:30.608244 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44198412-ad08-40b5-aacd-6c4ab79ecb34 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:30.608527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44198412-ad08-40b5-aacd-6c4ab79ecb34 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:30.608799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44198412-ad08-40b5-aacd-6c4ab79ecb34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:30.609464 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44198412-ad08-40b5-aacd-6c4ab79ecb34 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:30.610870 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44198412-ad08-40b5-aacd-6c4ab79ecb34 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:30.627469 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af9906-336a-4d1a-8586-63200f3c49a2 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:30.643582 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 524/1043] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:14:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:14:30.671531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a6f4d-0763-42ed-86a1-a9323c1507bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:30.672235 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a6f4d-0763-42ed-86a1-a9323c1507bf None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:30.672518 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a6f4d-0763-42ed-86a1-a9323c1507bf None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:30.672791 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a6f4d-0763-42ed-86a1-a9323c1507bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:30.682550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a6f4d-0763-42ed-86a1-a9323c1507bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:30.684114 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a6f4d-0763-42ed-86a1-a9323c1507bf None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:30.697879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-459a6f4d-0763-42ed-86a1-a9323c1507bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:14:30.697879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:30.697879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:30.697879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:30.697879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:30.697879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:30.697879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:30.697879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:30.697879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:30.697879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:30.697879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:30.697879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:30.697879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:14:30.697879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:14:30.697879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:14:30.697879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:30.698981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:30.700581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:30.700581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:30.700581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:14:30.700581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:14:30.700581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:14:30.700581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:14:30.700581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:14:30.700581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:14:30.700581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:14:30.700581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:30.700581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 520/1044] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Aug 7 19:14:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:30.705253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44198412-ad08-40b5-aacd-6c4ab79ecb34 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:30.819229 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44198412-ad08-40b5-aacd-6c4ab79ecb34 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:30.855031 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d25dc-b6f1-4989-8383-790fc4d8363a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:30.855651 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d25dc-b6f1-4989-8383-790fc4d8363a None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:30.855926 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d25dc-b6f1-4989-8383-790fc4d8363a None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:30.856230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d25dc-b6f1-4989-8383-790fc4d8363a None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:30.856969 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d25dc-b6f1-4989-8383-790fc4d8363a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:30.857272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d25dc-b6f1-4989-8383-790fc4d8363a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:30.858709 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3d25dc-b6f1-4989-8383-790fc4d8363a None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:30.877079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 525/1045] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Aug 7 19:14:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:30.882796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a6f4d-0763-42ed-86a1-a9323c1507bf None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:30.987600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a6f4d-0763-42ed-86a1-a9323c1507bf None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:31.020945 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:31.028019 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7739235-daed-4ac9-bcb1-f9af744cad93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:31.028640 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7739235-daed-4ac9-bcb1-f9af744cad93 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:31.028920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7739235-daed-4ac9-bcb1-f9af744cad93 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:31.029193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7739235-daed-4ac9-bcb1-f9af744cad93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:31.100506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7739235-daed-4ac9-bcb1-f9af744cad93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:31.101928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7739235-daed-4ac9-bcb1-f9af744cad93 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:31.210534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 521/1046] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:14:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:14:31.211802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d25dc-b6f1-4989-8383-790fc4d8363a None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:31.261705 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:31.299852 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c11cda-eb62-4990-8b88-432e2bde0b3f None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:31.300175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c11cda-eb62-4990-8b88-432e2bde0b3f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:31.300454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c11cda-eb62-4990-8b88-432e2bde0b3f None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:31.301822 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 522/1047] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:31.307429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dff0dc-0622-4613-bfef-5de6d6dfc0bb None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:31.307763 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dff0dc-0622-4613-bfef-5de6d6dfc0bb None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:31.308078 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dff0dc-0622-4613-bfef-5de6d6dfc0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:31.324291 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3d25dc-b6f1-4989-8383-790fc4d8363a None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:31.326558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-49dff0dc-0622-4613-bfef-5de6d6dfc0bb None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:31.327773 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-49dff0dc-0622-4613-bfef-5de6d6dfc0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:31.353912 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:31.366483 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446204c-2121-4b8b-b280-3f04da8ac561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:31.367110 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446204c-2121-4b8b-b280-3f04da8ac561 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:31.368047 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446204c-2121-4b8b-b280-3f04da8ac561 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:31.368047 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446204c-2121-4b8b-b280-3f04da8ac561 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:31.379044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446204c-2121-4b8b-b280-3f04da8ac561 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:31.380745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446204c-2121-4b8b-b280-3f04da8ac561 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:31.397270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-1446204c-2121-4b8b-b280-3f04da8ac561 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:31.397270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:31.397270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:31.397270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:31.397270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:31.397270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:31.397270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:31.397270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:31.397270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:31.397270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:31.397270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:31.397270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:31.397270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:31.397270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:31.397270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:31.397270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:31.398259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:31.398259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:31.398259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:31.398259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:31.398259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:31.398259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:31.398259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:31.398259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:31.398259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:31.398259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:31.398259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:31.398259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:31.398259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:31.398259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:31.398259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:31.398259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:31.398259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:31.399169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:31.399169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:31.399169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:31.399169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 526/1048] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:31.404312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446204c-2121-4b8b-b280-3f04da8ac561 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:31.412142 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 523/1049] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 19:14:31.430407 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dff0dc-0622-4613-bfef-5de6d6dfc0bb None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:31.506163 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:31.513310 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446204c-2121-4b8b-b280-3f04da8ac561 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:31.541133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dff0dc-0622-4613-bfef-5de6d6dfc0bb None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:31.550905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5328a-208c-4caa-9d03-18ab495edf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:31.551536 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5328a-208c-4caa-9d03-18ab495edf81 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:31.551968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5328a-208c-4caa-9d03-18ab495edf81 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:31.552464 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5328a-208c-4caa-9d03-18ab495edf81 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:31.553355 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5328a-208c-4caa-9d03-18ab495edf81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:31.553661 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5328a-208c-4caa-9d03-18ab495edf81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:31.555048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba5328a-208c-4caa-9d03-18ab495edf81 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:31.568365 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 527/1050] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:31.573967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5328a-208c-4caa-9d03-18ab495edf81 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:31.578985 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb1db0-306a-4f59-b5d7-4cbce63d751d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:31.579630 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb1db0-306a-4f59-b5d7-4cbce63d751d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:31.579964 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb1db0-306a-4f59-b5d7-4cbce63d751d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:31.580264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb1db0-306a-4f59-b5d7-4cbce63d751d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:31.591131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb1db0-306a-4f59-b5d7-4cbce63d751d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:14:31.595795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-89bb1db0-306a-4f59-b5d7-4cbce63d751d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:14:31.595795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:31.595795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:31.595795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:31.595795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:31.595795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:31.595795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:31.595795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:31.595795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:31.595795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:31.595795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:31.595795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:31.595795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:14:31.595795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:14:31.595795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:14:31.595795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:14:31.596885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:14:31.596885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:14:31.596885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:14:31.596885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:14:31.596885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:31.596885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:31.596885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:31.596885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:31.596885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:31.596885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:31.596885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:31.596885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:31.596885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:31.596885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:31.596885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:31.596885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:31.597768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:31.597768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:31.597768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:31.597768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:31.597768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:14:31.597768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:14:31.597768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:14:31.597768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:14:31.597768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:14:31.597768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:14:31.597768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:14:31.597768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:31.597768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 524/1051] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:14:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:31.602929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb1db0-306a-4f59-b5d7-4cbce63d751d None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:31.626626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:31.685220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba5328a-208c-4caa-9d03-18ab495edf81 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:31.707231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb1db0-306a-4f59-b5d7-4cbce63d751d None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:31.722361 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d2ec3-dab6-4b17-bf0c-a621bca0ab1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:31.723052 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d2ec3-dab6-4b17-bf0c-a621bca0ab1e None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:31.723357 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d2ec3-dab6-4b17-bf0c-a621bca0ab1e None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:31.723644 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d2ec3-dab6-4b17-bf0c-a621bca0ab1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:31.730576 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d2ec3-dab6-4b17-bf0c-a621bca0ab1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:31.732062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d2ec3-dab6-4b17-bf0c-a621bca0ab1e None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:31.743174 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5019102-d151-4471-bd62-e94e7d46aaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:31.743627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-f47d2ec3-dab6-4b17-bf0c-a621bca0ab1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:31.743627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:31.743627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:31.743627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:31.743627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:31.743627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:31.743627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:31.743627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:31.743627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:31.743627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:31.743627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:31.743627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:31.743627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:14:31.743627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:14:31.743627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:14:31.744692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:14:31.744692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:31.744692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:31.744692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:31.744692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:31.744692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:31.744692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:31.744692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:31.744692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:31.744692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:31.744692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:31.744692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:31.744692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:31.744692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:31.744692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:31.744692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:31.744692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:31.745652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:14:31.745652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:14:31.745652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:14:31.745652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:14:31.745652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:14:31.745652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:14:31.745652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:31.745652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:31.746089 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5019102-d151-4471-bd62-e94e7d46aaf8 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:31.746089 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5019102-d151-4471-bd62-e94e7d46aaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:31.746089 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5019102-d151-4471-bd62-e94e7d46aaf8 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:31.746280 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 528/1052] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:14:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:31.749863 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5019102-d151-4471-bd62-e94e7d46aaf8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:31.750237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5019102-d151-4471-bd62-e94e7d46aaf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:31.750879 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d2ec3-dab6-4b17-bf0c-a621bca0ab1e None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:31.751749 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5019102-d151-4471-bd62-e94e7d46aaf8 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:31.765376 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 525/1053] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:14:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:31.766563 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5019102-d151-4471-bd62-e94e7d46aaf8 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:31.772896 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:31.853175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:31.864140 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d2ec3-dab6-4b17-bf0c-a621bca0ab1e None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:31.877299 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5019102-d151-4471-bd62-e94e7d46aaf8 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:31.897789 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d0e67-c2fd-4440-9745-c2b66ceccb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:31.898394 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d0e67-c2fd-4440-9745-c2b66ceccb4f None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:31.898673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d0e67-c2fd-4440-9745-c2b66ceccb4f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:31.898951 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d0e67-c2fd-4440-9745-c2b66ceccb4f None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:31.899649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d0e67-c2fd-4440-9745-c2b66ceccb4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:31.899955 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d0e67-c2fd-4440-9745-c2b66ceccb4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:31.901512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d0e67-c2fd-4440-9745-c2b66ceccb4f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:31.913532 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 529/1054] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:14:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:31.919780 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d0e67-c2fd-4440-9745-c2b66ceccb4f None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:31.926869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efac78-d3c2-479c-b170-093d53759044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:31.928184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efac78-d3c2-479c-b170-093d53759044 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:31.928788 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efac78-d3c2-479c-b170-093d53759044 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:31.929067 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efac78-d3c2-479c-b170-093d53759044 None admin] PATH_INFO: `/v3/users/b6f583f1e37242c6a51b12dd4f1ff1e6` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:31.943832 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efac78-d3c2-479c-b170-093d53759044 None admin] RBAC: Authorizing `identity:get_user(user_id=b6f583f1e37242c6a51b12dd4f1ff1e6)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:31.945316 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efac78-d3c2-479c-b170-093d53759044 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:31.946982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 526/1055] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:14:31 2022] GET /identity/v3/users/b6f583f1e37242c6a51b12dd4f1ff1e6 => generated 285 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:31.952899 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efac78-d3c2-479c-b170-093d53759044 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:32.002032 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:32.033667 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d0e67-c2fd-4440-9745-c2b66ceccb4f None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:32.062015 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efac78-d3c2-479c-b170-093d53759044 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:32.069016 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87516461-460a-4b08-8471-49eb555d2a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:32.069627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87516461-460a-4b08-8471-49eb555d2a79 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:32.069963 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87516461-460a-4b08-8471-49eb555d2a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:32.070238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87516461-460a-4b08-8471-49eb555d2a79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:32.070984 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87516461-460a-4b08-8471-49eb555d2a79 None admin] RBAC: Adding query filter params (role.id=c5946f310572439687baeed49acd4336, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=a4946aec299f45bd8402cff3159d04d7) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:32.071286 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87516461-460a-4b08-8471-49eb555d2a79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:32.072756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87516461-460a-4b08-8471-49eb555d2a79 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:32.078416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 530/1056] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:14:31 2022] GET /identity/v3/role_assignments?user.id=a4946aec299f45bd8402cff3159d04d7&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=c5946f310572439687baeed49acd4336 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:32.099940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9158fe-b369-4dac-85a0-80b8f28ee5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:32.100625 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9158fe-b369-4dac-85a0-80b8f28ee5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:32.100901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9158fe-b369-4dac-85a0-80b8f28ee5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:32.101196 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9158fe-b369-4dac-85a0-80b8f28ee5c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:32.114314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:32.181571 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9158fe-b369-4dac-85a0-80b8f28ee5c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:32.183463 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9158fe-b369-4dac-85a0-80b8f28ee5c6 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:32.295515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 527/1057] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:14:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 07 19:14:32.301629 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87516461-460a-4b08-8471-49eb555d2a79 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:32.411053 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87516461-460a-4b08-8471-49eb555d2a79 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:32.447143 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414c48f-d6f0-4e1b-b50d-7d8d38bc61b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:32.447691 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414c48f-d6f0-4e1b-b50d-7d8d38bc61b5 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:32.447958 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414c48f-d6f0-4e1b-b50d-7d8d38bc61b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:32.448279 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414c48f-d6f0-4e1b-b50d-7d8d38bc61b5 None admin] PATH_INFO: `/v3/projects/f2e6c2ad22784bd7807c964bcda423d1` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:32.449417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414c48f-d6f0-4e1b-b50d-7d8d38bc61b5 None admin] RBAC: Authorizing `identity:get_project(project_id=f2e6c2ad22784bd7807c964bcda423d1)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:32.450863 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414c48f-d6f0-4e1b-b50d-7d8d38bc61b5 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:32.457501 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 531/1058] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:14:32 2022] GET /identity/v3/projects/f2e6c2ad22784bd7807c964bcda423d1 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:32.458742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9158fe-b369-4dac-85a0-80b8f28ee5c6 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:32.501702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:32.562140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9158fe-b369-4dac-85a0-80b8f28ee5c6 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:32.597605 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81cef6-c9c6-48be-a41d-ec39bb4bbdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:32.598215 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81cef6-c9c6-48be-a41d-ec39bb4bbdb1 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:32.598488 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81cef6-c9c6-48be-a41d-ec39bb4bbdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:32.598752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81cef6-c9c6-48be-a41d-ec39bb4bbdb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:32.599473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81cef6-c9c6-48be-a41d-ec39bb4bbdb1 None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=f2e6c2ad22784bd7807c964bcda423d1, user.id=b6f583f1e37242c6a51b12dd4f1ff1e6) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:32.599778 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81cef6-c9c6-48be-a41d-ec39bb4bbdb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:32.601230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81cef6-c9c6-48be-a41d-ec39bb4bbdb1 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:32.615765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 528/1059] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:14:32 2022] GET /identity/v3/role_assignments?user.id=b6f583f1e37242c6a51b12dd4f1ff1e6&scope.project.id=f2e6c2ad22784bd7807c964bcda423d1&role.id=4da42f08d107448fb877b42b61bc74ea => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:32.692036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9077c3-34f9-4729-b4c9-4b44ef88c95f None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:32.692282 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9077c3-34f9-4729-b4c9-4b44ef88c95f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:32.692819 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9077c3-34f9-4729-b4c9-4b44ef88c95f None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:32.694247 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 532/1060] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:32.700381 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c24957-de0e-4634-8c66-1b6ee0286ceb None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:32.700659 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c24957-de0e-4634-8c66-1b6ee0286ceb None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:32.700872 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c24957-de0e-4634-8c66-1b6ee0286ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:32.720469 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-d2c24957-de0e-4634-8c66-1b6ee0286ceb None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:32.721709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d2c24957-de0e-4634-8c66-1b6ee0286ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:32.806330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 529/1061] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:32.828669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9077c3-34f9-4729-b4c9-4b44ef88c95f None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:32.841432 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:32.942495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9077c3-34f9-4729-b4c9-4b44ef88c95f None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:32.977646 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa224ff-9c2e-4640-bd0d-94e7027ba49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:32.978189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa224ff-9c2e-4640-bd0d-94e7027ba49c None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:32.978467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa224ff-9c2e-4640-bd0d-94e7027ba49c None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:32.978739 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa224ff-9c2e-4640-bd0d-94e7027ba49c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:33.058092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa224ff-9c2e-4640-bd0d-94e7027ba49c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:33.059556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa224ff-9c2e-4640-bd0d-94e7027ba49c None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:33.069624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:33.161333 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 533/1062] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:14:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:14:33.167277 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c24957-de0e-4634-8c66-1b6ee0286ceb None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:33.207058 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:33.278424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c24957-de0e-4634-8c66-1b6ee0286ceb None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:33.313745 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b1989-856c-40b5-897d-f6f1d71b6037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:33.314388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b1989-856c-40b5-897d-f6f1d71b6037 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:33.314659 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b1989-856c-40b5-897d-f6f1d71b6037 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:33.314937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b1989-856c-40b5-897d-f6f1d71b6037 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:33.325643 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b1989-856c-40b5-897d-f6f1d71b6037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:33.327177 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b1989-856c-40b5-897d-f6f1d71b6037 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:33.342278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-1d2b1989-856c-40b5-897d-f6f1d71b6037 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:33.342278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:33.342278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:33.342278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:33.342278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:33.342278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:33.342278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:33.342278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:33.342278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:33.342278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:33.342278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:33.342278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:33.342278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:33.342278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:33.342278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:33.342278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:33.343282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:33.343282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:33.343282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:33.343282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:33.343282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:33.343282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:33.343282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:33.343282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:33.343282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:33.343282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:33.343282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:33.343282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:33.343282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:33.343282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:33.343282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:33.343282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:33.343282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:33.344213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:33.344213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:33.344213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:33.348560 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 530/1063] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:33.349603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa224ff-9c2e-4640-bd0d-94e7027ba49c None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:33.448723 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa224ff-9c2e-4640-bd0d-94e7027ba49c None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:33.457974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:33.484412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf487f-9153-40f7-926a-d36da9fa6d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:33.485031 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf487f-9153-40f7-926a-d36da9fa6d0f None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:33.485312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf487f-9153-40f7-926a-d36da9fa6d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:33.485584 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf487f-9153-40f7-926a-d36da9fa6d0f None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:33.486227 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf487f-9153-40f7-926a-d36da9fa6d0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:33.486529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf487f-9153-40f7-926a-d36da9fa6d0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:33.487911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf487f-9153-40f7-926a-d36da9fa6d0f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:33.508129 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 534/1064] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:34.071263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f59505-5da2-4dad-829f-ef283c764d18 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:34.071840 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f59505-5da2-4dad-829f-ef283c764d18 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:34.072165 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f59505-5da2-4dad-829f-ef283c764d18 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:34.073535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 531/1065] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:34.078788 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854092a0-a8fe-4fb3-89f4-3deac73cfa80 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:34.079277 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854092a0-a8fe-4fb3-89f4-3deac73cfa80 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:34.079277 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854092a0-a8fe-4fb3-89f4-3deac73cfa80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:34.097946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-854092a0-a8fe-4fb3-89f4-3deac73cfa80 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:34.099120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-854092a0-a8fe-4fb3-89f4-3deac73cfa80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:34.168727 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 535/1066] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:34.191188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f59505-5da2-4dad-829f-ef283c764d18 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:34.280720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:34.298338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f59505-5da2-4dad-829f-ef283c764d18 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:34.334603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f137c-afe7-4e90-ac1e-50bc6afd5c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:34.335240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f137c-afe7-4e90-ac1e-50bc6afd5c48 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:34.335522 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f137c-afe7-4e90-ac1e-50bc6afd5c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:34.335798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f137c-afe7-4e90-ac1e-50bc6afd5c48 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:34.346710 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f137c-afe7-4e90-ac1e-50bc6afd5c48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:14:34.351408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-f51f137c-afe7-4e90-ac1e-50bc6afd5c48 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 19:14:34.351408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:34.351408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:34.351408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:34.351408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:34.351408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:34.351408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:34.351408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:34.351408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:34.351408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:34.351408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:34.351408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:34.351408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:14:34.351408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:14:34.351408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:14:34.351408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:14:34.352542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:14:34.352542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:14:34.352542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:14:34.352542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:14:34.352542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:34.352542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:34.352542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:34.352542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:34.352542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:34.352542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:34.352542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:34.352542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:34.352542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:34.352542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:34.352542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:34.352542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:34.353370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:34.353370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:34.353370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:34.353370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:34.353370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:14:34.353370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:14:34.353370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:14:34.353370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:14:34.353370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:14:34.353370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:14:34.353370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 19:14:34.353370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:34.353370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 532/1067] 10.222.0.17 () {56 vars in 1120 bytes} [Sun Aug 7 19:14:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:34.358905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854092a0-a8fe-4fb3-89f4-3deac73cfa80 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:34.459256 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:34.463577 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854092a0-a8fe-4fb3-89f4-3deac73cfa80 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:34.501845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689b64d-8f7c-4ca1-a140-191b4cf00a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:34.502464 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689b64d-8f7c-4ca1-a140-191b4cf00a7a None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:34.502743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689b64d-8f7c-4ca1-a140-191b4cf00a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:34.503017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689b64d-8f7c-4ca1-a140-191b4cf00a7a None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:34.503694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689b64d-8f7c-4ca1-a140-191b4cf00a7a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:34.503967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689b64d-8f7c-4ca1-a140-191b4cf00a7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:34.505390 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689b64d-8f7c-4ca1-a140-191b4cf00a7a None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:34.514655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 536/1068] 10.222.0.17 () {56 vars in 1115 bytes} [Sun Aug 7 19:14:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:34.520946 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f137c-afe7-4e90-ac1e-50bc6afd5c48 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:34.598036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b2172-557c-4ba2-bb8a-30e1ebe5331b None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:34.598262 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b2172-557c-4ba2-bb8a-30e1ebe5331b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:34.598469 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b2172-557c-4ba2-bb8a-30e1ebe5331b None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:34.599806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 537/1069] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:34.605126 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565b17e-4547-45e3-8619-583676ed88e5 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:34.605407 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565b17e-4547-45e3-8619-583676ed88e5 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:34.605700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565b17e-4547-45e3-8619-583676ed88e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:34.624346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-0565b17e-4547-45e3-8619-583676ed88e5 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:34.625579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0565b17e-4547-45e3-8619-583676ed88e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:34.626681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f137c-afe7-4e90-ac1e-50bc6afd5c48 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:34.662672 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9629df6-0cfb-4e83-8651-43aee80a289d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:34.663303 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9629df6-0cfb-4e83-8651-43aee80a289d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:34.663583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9629df6-0cfb-4e83-8651-43aee80a289d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:34.663853 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9629df6-0cfb-4e83-8651-43aee80a289d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:34.674698 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9629df6-0cfb-4e83-8651-43aee80a289d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:34.676326 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9629df6-0cfb-4e83-8651-43aee80a289d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:34.684067 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:34.693446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-b9629df6-0cfb-4e83-8651-43aee80a289d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:34.693446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:34.693446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:34.693446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:34.693446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:34.693446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:34.693446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:34.693446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:34.693446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:34.693446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:34.693446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:34.693446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:34.693446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:34.693446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:34.693446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:34.693446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:34.694529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:34.694529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:34.694529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:34.694529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:34.694529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:34.694529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:34.694529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:34.694529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:34.694529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:34.694529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:34.694529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:34.694529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:34.694529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:34.694529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:34.694529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:34.694529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:34.694529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:34.695436 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:34.695436 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:34.695436 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:34.695436 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 533/1070] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:34.698128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 538/1071] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:34.700711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9629df6-0cfb-4e83-8651-43aee80a289d None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:34.720104 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565b17e-4547-45e3-8619-583676ed88e5 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:34.802928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9629df6-0cfb-4e83-8651-43aee80a289d None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:34.812918 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:34.823465 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565b17e-4547-45e3-8619-583676ed88e5 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:34.838400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a464c37-a81e-4c96-ab89-a40eae32cfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:34.839067 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a464c37-a81e-4c96-ab89-a40eae32cfb9 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:34.839377 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a464c37-a81e-4c96-ab89-a40eae32cfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:34.839625 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a464c37-a81e-4c96-ab89-a40eae32cfb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:34.840308 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a464c37-a81e-4c96-ab89-a40eae32cfb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:34.840613 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a464c37-a81e-4c96-ab89-a40eae32cfb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:34.841992 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a464c37-a81e-4c96-ab89-a40eae32cfb9 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:34.854467 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 534/1072] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:34.858256 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7ca74-86b7-454a-a056-26acf52465e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:34.858901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7ca74-86b7-454a-a056-26acf52465e2 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:34.859450 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7ca74-86b7-454a-a056-26acf52465e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:34.859722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7ca74-86b7-454a-a056-26acf52465e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:34.860131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a464c37-a81e-4c96-ab89-a40eae32cfb9 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:34.870218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7ca74-86b7-454a-a056-26acf52465e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:14:34.874907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-dad7ca74-86b7-454a-a056-26acf52465e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:14:34.874907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:34.874907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:34.874907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:34.874907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:34.874907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:34.874907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:34.874907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:34.874907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:34.874907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:34.874907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:34.874907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:34.874907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:14:34.874907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:14:34.874907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:14:34.874907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:14:34.876081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:14:34.876081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:14:34.876081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:14:34.876081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:14:34.876081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:34.876081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:34.876081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:34.876081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:34.876081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:34.876081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:34.876081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:34.876081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:34.876081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:34.876081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:34.876081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:34.876081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:34.877127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:34.877127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:34.877127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:34.877127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:34.877127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:14:34.877127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:14:34.877127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:14:34.877127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:14:34.877127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:14:34.877127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:14:34.877127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:14:34.877127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:34.877127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 539/1073] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:14:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:34.882119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7ca74-86b7-454a-a056-26acf52465e2 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:34.924512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:34.971327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a464c37-a81e-4c96-ab89-a40eae32cfb9 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:34.997305 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7ca74-86b7-454a-a056-26acf52465e2 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:35.005710 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d53e46-5560-472f-8fd4-354bca519aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:35.006389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d53e46-5560-472f-8fd4-354bca519aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:35.006653 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d53e46-5560-472f-8fd4-354bca519aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:35.006927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d53e46-5560-472f-8fd4-354bca519aa5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:35.014513 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d53e46-5560-472f-8fd4-354bca519aa5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:35.015969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d53e46-5560-472f-8fd4-354bca519aa5 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:35.027836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-69d53e46-5560-472f-8fd4-354bca519aa5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:14:35.027836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:35.027836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:35.027836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:35.027836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:35.027836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:35.027836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:35.027836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:35.027836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:35.027836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:35.027836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:35.027836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:35.027836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:14:35.027836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:14:35.027836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:14:35.027836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:35.028825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:35.029752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:35.029752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:35.029752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:14:35.029752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:14:35.029752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:14:35.029752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:14:35.029752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:14:35.029752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:14:35.029752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:14:35.029752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:35.029752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 535/1074] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Aug 7 19:14:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:35.034851 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d53e46-5560-472f-8fd4-354bca519aa5 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:35.039566 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f87e33-1955-42ae-9087-9ea4cdcacc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:35.040220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f87e33-1955-42ae-9087-9ea4cdcacc60 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:35.040496 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f87e33-1955-42ae-9087-9ea4cdcacc60 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:35.040769 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f87e33-1955-42ae-9087-9ea4cdcacc60 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:35.041443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f87e33-1955-42ae-9087-9ea4cdcacc60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:35.041745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f87e33-1955-42ae-9087-9ea4cdcacc60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:35.043181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f87e33-1955-42ae-9087-9ea4cdcacc60 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:35.051588 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 540/1075] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:14:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:35.057975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f87e33-1955-42ae-9087-9ea4cdcacc60 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:35.073214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:35.132482 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d53e46-5560-472f-8fd4-354bca519aa5 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:35.149850 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:35.158970 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f87e33-1955-42ae-9087-9ea4cdcacc60 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:35.167781 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c7b53-850f-4dcd-b7ee-934efd51d7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:35.168419 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c7b53-850f-4dcd-b7ee-934efd51d7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:35.168709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c7b53-850f-4dcd-b7ee-934efd51d7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:35.168985 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c7b53-850f-4dcd-b7ee-934efd51d7d1 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:35.169715 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c7b53-850f-4dcd-b7ee-934efd51d7d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:35.170021 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c7b53-850f-4dcd-b7ee-934efd51d7d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:35.171458 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c7b53-850f-4dcd-b7ee-934efd51d7d1 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:35.190044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 536/1076] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Aug 7 19:14:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:35.193137 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30562b0-4180-43e0-8fea-fa8b16065dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:35.193776 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30562b0-4180-43e0-8fea-fa8b16065dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:35.194048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30562b0-4180-43e0-8fea-fa8b16065dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:35.194315 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30562b0-4180-43e0-8fea-fa8b16065dd5 None admin] PATH_INFO: `/v3/users/b6f583f1e37242c6a51b12dd4f1ff1e6` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:35.195535 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30562b0-4180-43e0-8fea-fa8b16065dd5 None admin] RBAC: Authorizing `identity:get_user(user_id=b6f583f1e37242c6a51b12dd4f1ff1e6)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:35.195819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c7b53-850f-4dcd-b7ee-934efd51d7d1 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:35.197007 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30562b0-4180-43e0-8fea-fa8b16065dd5 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:35.203406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 541/1077] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:14:35 2022] GET /identity/v3/users/b6f583f1e37242c6a51b12dd4f1ff1e6 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:35.204449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30562b0-4180-43e0-8fea-fa8b16065dd5 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:35.290436 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:35.295898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c7b53-850f-4dcd-b7ee-934efd51d7d1 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:35.314924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30562b0-4180-43e0-8fea-fa8b16065dd5 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:35.336696 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789af1e4-dbe0-41d2-8f54-104e5a293a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:35.337334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789af1e4-dbe0-41d2-8f54-104e5a293a49 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:35.337607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789af1e4-dbe0-41d2-8f54-104e5a293a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:35.337873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789af1e4-dbe0-41d2-8f54-104e5a293a49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:35.348139 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789af1e4-dbe0-41d2-8f54-104e5a293a49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:35.349600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789af1e4-dbe0-41d2-8f54-104e5a293a49 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:35.355051 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534eaf98-496b-4259-966c-446a206cac5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:35.355686 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534eaf98-496b-4259-966c-446a206cac5b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:35.355964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534eaf98-496b-4259-966c-446a206cac5b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:35.356270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534eaf98-496b-4259-966c-446a206cac5b None admin] PATH_INFO: `/v3/projects/f2e6c2ad22784bd7807c964bcda423d1` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:35.357395 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534eaf98-496b-4259-966c-446a206cac5b None admin] RBAC: Authorizing `identity:get_project(project_id=f2e6c2ad22784bd7807c964bcda423d1)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:35.358834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534eaf98-496b-4259-966c-446a206cac5b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:35.360530 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 542/1078] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:14:35 2022] GET /identity/v3/projects/f2e6c2ad22784bd7807c964bcda423d1 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:35.364100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-789af1e4-dbe0-41d2-8f54-104e5a293a49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:35.364100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:35.364100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:35.364100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:35.364100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:35.364100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:35.364100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:35.364100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:35.364100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:35.364100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:35.364100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:35.364100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:35.364100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:35.364100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:35.364100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:35.364100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:35.365253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:35.365253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:35.365253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:35.365253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:35.365253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:35.365253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:35.365253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:35.365253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:35.365253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:35.365253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:35.365253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:35.365253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:35.365253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:35.365253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:35.365253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:35.365253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:35.365253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:35.366389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:35.366389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:35.366389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:35.366389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 537/1079] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:35.370919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534eaf98-496b-4259-966c-446a206cac5b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:35.371055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789af1e4-dbe0-41d2-8f54-104e5a293a49 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:35.400669 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:35.472553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534eaf98-496b-4259-966c-446a206cac5b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:35.476482 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789af1e4-dbe0-41d2-8f54-104e5a293a49 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:35.507891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca50de6-a010-41b2-b4d3-a2a8104a87f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:35.508540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50de6-a010-41b2-b4d3-a2a8104a87f1 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:35.508823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50de6-a010-41b2-b4d3-a2a8104a87f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:35.509099 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50de6-a010-41b2-b4d3-a2a8104a87f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:35.509746 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca50de6-a010-41b2-b4d3-a2a8104a87f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:35.510053 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca50de6-a010-41b2-b4d3-a2a8104a87f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:35.511045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e42c7d-16b9-4139-aee5-455efdfc60eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:35.511436 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca50de6-a010-41b2-b4d3-a2a8104a87f1 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:35.511711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e42c7d-16b9-4139-aee5-455efdfc60eb None admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:35.512017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e42c7d-16b9-4139-aee5-455efdfc60eb None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:35.512290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e42c7d-16b9-4139-aee5-455efdfc60eb None admin] PATH_INFO: `/v3/projects/f2e6c2ad22784bd7807c964bcda423d1/users/b6f583f1e37242c6a51b12dd4f1ff1e6/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:35.514205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e42c7d-16b9-4139-aee5-455efdfc60eb None admin] RBAC: Authorizing `identity:create_grant(project_id=f2e6c2ad22784bd7807c964bcda423d1, user_id=b6f583f1e37242c6a51b12dd4f1ff1e6, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:35.515624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e42c7d-16b9-4139-aee5-455efdfc60eb None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:35.524455 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 543/1080] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:35.529751 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 538/1081] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:14:35 2022] PUT /identity/v3/projects/f2e6c2ad22784bd7807c964bcda423d1/users/b6f583f1e37242c6a51b12dd4f1ff1e6/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:14:35.530409 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca50de6-a010-41b2-b4d3-a2a8104a87f1 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:35.542205 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:35.641747 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca50de6-a010-41b2-b4d3-a2a8104a87f1 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:35.683565 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7228e35-6eed-4d04-b64c-b46a745c0ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:35.684673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7228e35-6eed-4d04-b64c-b46a745c0ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:35.684673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7228e35-6eed-4d04-b64c-b46a745c0ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:35.684933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7228e35-6eed-4d04-b64c-b46a745c0ec8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:35.692961 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7228e35-6eed-4d04-b64c-b46a745c0ec8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:35.694467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7228e35-6eed-4d04-b64c-b46a745c0ec8 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:35.707014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-d7228e35-6eed-4d04-b64c-b46a745c0ec8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:35.707014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:35.707014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:35.707014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:35.707014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:35.707014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:35.707014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:35.707014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:35.707014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:35.707014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:35.707014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:35.707014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:35.707014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:14:35.707014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:14:35.707014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:14:35.708067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:14:35.708067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:35.708067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:35.708067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:35.708067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:35.708067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:35.708067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:35.708067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:35.708067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:35.708067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:35.708067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:35.708067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:35.708067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:35.708067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:35.708067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:35.708067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:35.708067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:35.708962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:14:35.708962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:14:35.708962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:14:35.708962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:14:35.708962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:14:35.708962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:14:35.708962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:35.708962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:35.708962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 544/1082] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:14:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:35.714550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e42c7d-16b9-4139-aee5-455efdfc60eb None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:35.805454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:35.818984 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e42c7d-16b9-4139-aee5-455efdfc60eb None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:35.854362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6f97e-b7d0-48de-8694-79b4e3d7b3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:35.855028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6f97e-b7d0-48de-8694-79b4e3d7b3ec None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:35.855304 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6f97e-b7d0-48de-8694-79b4e3d7b3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:35.855574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6f97e-b7d0-48de-8694-79b4e3d7b3ec None admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:35.856325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6f97e-b7d0-48de-8694-79b4e3d7b3ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:35.856620 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6f97e-b7d0-48de-8694-79b4e3d7b3ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:35.858146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6f97e-b7d0-48de-8694-79b4e3d7b3ec None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:35.870361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 539/1083] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:14:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:35.876433 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7228e35-6eed-4d04-b64c-b46a745c0ec8 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:35.946092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:35.983346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7228e35-6eed-4d04-b64c-b46a745c0ec8 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:36.019369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa87bc9-3b14-4bb5-9f57-41c8ce054648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:36.020078 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa87bc9-3b14-4bb5-9f57-41c8ce054648 None admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:36.020317 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa87bc9-3b14-4bb5-9f57-41c8ce054648 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:36.020620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa87bc9-3b14-4bb5-9f57-41c8ce054648 None admin] PATH_INFO: `/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/a4946aec299f45bd8402cff3159d04d7/roles/c5946f310572439687baeed49acd4336` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:36.022816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa87bc9-3b14-4bb5-9f57-41c8ce054648 None admin] RBAC: Authorizing `identity:create_grant(project_id=37703d50cd7e4d9bbdc797a7a1466b66, user_id=a4946aec299f45bd8402cff3159d04d7, role_id=c5946f310572439687baeed49acd4336)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:36.024299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa87bc9-3b14-4bb5-9f57-41c8ce054648 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:36.039543 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 545/1084] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:14:35 2022] PUT /identity/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/a4946aec299f45bd8402cff3159d04d7/roles/c5946f310572439687baeed49acd4336 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:14:37.528339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb627e-70aa-41f5-8302-342dbb82f347 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:37.528881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb627e-70aa-41f5-8302-342dbb82f347 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:37.528881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb627e-70aa-41f5-8302-342dbb82f347 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:37.530233 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 540/1085] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:37.535584 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1704f9-ceaf-42e3-ae04-e956d47a48e4 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:37.536049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1704f9-ceaf-42e3-ae04-e956d47a48e4 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:37.536136 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1704f9-ceaf-42e3-ae04-e956d47a48e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:37.554580 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-5d1704f9-ceaf-42e3-ae04-e956d47a48e4 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:37.555766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-5d1704f9-ceaf-42e3-ae04-e956d47a48e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:37.624918 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 546/1086] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:37.648572 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb627e-70aa-41f5-8302-342dbb82f347 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:37.755706 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb627e-70aa-41f5-8302-342dbb82f347 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:37.791045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d002ba8-59bd-4579-9207-b4658d015fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:37.791587 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d002ba8-59bd-4579-9207-b4658d015fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:37.791866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d002ba8-59bd-4579-9207-b4658d015fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:37.792197 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d002ba8-59bd-4579-9207-b4658d015fb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:37.802637 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d002ba8-59bd-4579-9207-b4658d015fb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:14:37.807676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-4d002ba8-59bd-4579-9207-b4658d015fb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:14:37.807676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:37.807676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:37.807676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:37.807676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:37.807676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:37.807676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:37.807676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:37.807676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:37.807676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:37.807676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:37.807676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:37.807676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:14:37.807676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:14:37.807676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:14:37.807676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:14:37.808783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:14:37.808783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:14:37.808783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:14:37.808783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:14:37.808783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:37.808783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:37.808783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:37.808783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:37.808783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:37.808783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:37.808783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:37.808783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:37.808783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:37.808783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:37.808783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:37.808783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:37.809656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:37.809656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:37.809656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:37.809656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:37.809656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:14:37.809656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:14:37.809656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:14:37.809656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:14:37.809656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:14:37.809656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:14:37.809656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:14:37.809656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:37.809656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 541/1087] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:14:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:37.815224 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1704f9-ceaf-42e3-ae04-e956d47a48e4 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:37.875564 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:37.919737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1704f9-ceaf-42e3-ae04-e956d47a48e4 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:37.960103 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95449b40-c00c-40cd-b5a7-d6d3cd2cb723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:37.960717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95449b40-c00c-40cd-b5a7-d6d3cd2cb723 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:37.960994 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95449b40-c00c-40cd-b5a7-d6d3cd2cb723 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:37.961264 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95449b40-c00c-40cd-b5a7-d6d3cd2cb723 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:37.961933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95449b40-c00c-40cd-b5a7-d6d3cd2cb723 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:37.962232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95449b40-c00c-40cd-b5a7-d6d3cd2cb723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:37.963619 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95449b40-c00c-40cd-b5a7-d6d3cd2cb723 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:37.972215 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 547/1088] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:14:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:37.978131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d002ba8-59bd-4579-9207-b4658d015fb8 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:37.989717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:37.996253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04711f87-6644-4526-a2a0-ecca76b99e46 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:37.996523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04711f87-6644-4526-a2a0-ecca76b99e46 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:37.996823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04711f87-6644-4526-a2a0-ecca76b99e46 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:37.998153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 548/1089] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:14:38.003114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cb05b-2e9e-4dcd-a6ee-58fb690d1dc1 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:38.003403 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cb05b-2e9e-4dcd-a6ee-58fb690d1dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:38.003676 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cb05b-2e9e-4dcd-a6ee-58fb690d1dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:38.022134 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-226cb05b-2e9e-4dcd-a6ee-58fb690d1dc1 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:38.023333 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-226cb05b-2e9e-4dcd-a6ee-58fb690d1dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:38.085078 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d002ba8-59bd-4579-9207-b4658d015fb8 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:38.091737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 549/1090] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 19:14:38.114689 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cb05b-2e9e-4dcd-a6ee-58fb690d1dc1 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:38.118823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a9dc4-99df-421d-a984-138870618d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:38.119463 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a9dc4-99df-421d-a984-138870618d7e None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:38.119745 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a9dc4-99df-421d-a984-138870618d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:38.120044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a9dc4-99df-421d-a984-138870618d7e None admin] PATH_INFO: `/v3/users/b6f583f1e37242c6a51b12dd4f1ff1e6` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:38.121278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a9dc4-99df-421d-a984-138870618d7e None admin] RBAC: Authorizing `identity:get_user(user_id=b6f583f1e37242c6a51b12dd4f1ff1e6)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:38.122696 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a9dc4-99df-421d-a984-138870618d7e None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:38.124369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 542/1091] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:14:37 2022] GET /identity/v3/users/b6f583f1e37242c6a51b12dd4f1ff1e6 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:38.130066 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a9dc4-99df-421d-a984-138870618d7e None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:38.163206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:38.210531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:38.212421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cb05b-2e9e-4dcd-a6ee-58fb690d1dc1 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:38.236548 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a9dc4-99df-421d-a984-138870618d7e None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:38.249993 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab6afa-f3e0-4f17-8704-3d69c8de8649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:38.250649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab6afa-f3e0-4f17-8704-3d69c8de8649 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:38.250924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab6afa-f3e0-4f17-8704-3d69c8de8649 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:38.251196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ab6afa-f3e0-4f17-8704-3d69c8de8649 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:38.262536 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ab6afa-f3e0-4f17-8704-3d69c8de8649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:14:38.267358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-42ab6afa-f3e0-4f17-8704-3d69c8de8649 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 19:14:38.267358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:38.267358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:38.267358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:38.267358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:38.267358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:38.267358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:38.267358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:38.267358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:38.267358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:38.267358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:38.267358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:38.267358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:14:38.267358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:14:38.267358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:14:38.267358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:14:38.268341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:14:38.268341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:14:38.268341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:14:38.268341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:14:38.268341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:38.268341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:38.268341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:38.268341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:38.268341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:38.268341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:38.268341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:38.268341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:38.268341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:38.268341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:38.268341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:38.268341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:38.269175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:38.269175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:38.269175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:38.269175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:38.269175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:14:38.269175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:14:38.269175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:14:38.269175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:14:38.269175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:14:38.269175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:14:38.269175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 19:14:38.269175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:38.269175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 550/1092] 10.222.0.17 () {56 vars in 1120 bytes} [Sun Aug 7 19:14:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:38.273581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7bb1f1-6f40-4a6e-b846-32a26aabd654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:38.274222 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bb1f1-6f40-4a6e-b846-32a26aabd654 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:38.274499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bb1f1-6f40-4a6e-b846-32a26aabd654 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:38.274764 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7bb1f1-6f40-4a6e-b846-32a26aabd654 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:38.274879 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab6afa-f3e0-4f17-8704-3d69c8de8649 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:38.341539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7bb1f1-6f40-4a6e-b846-32a26aabd654 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:38.342943 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7bb1f1-6f40-4a6e-b846-32a26aabd654 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:38.373654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ab6afa-f3e0-4f17-8704-3d69c8de8649 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:38.385676 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:38.411246 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c9a47-732d-4ac7-b2dc-57723dfd80ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:38.411864 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c9a47-732d-4ac7-b2dc-57723dfd80ca None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:38.412166 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c9a47-732d-4ac7-b2dc-57723dfd80ca None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:38.412433 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c9a47-732d-4ac7-b2dc-57723dfd80ca None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:38.413103 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c9a47-732d-4ac7-b2dc-57723dfd80ca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:38.413406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c9a47-732d-4ac7-b2dc-57723dfd80ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:38.414793 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c9a47-732d-4ac7-b2dc-57723dfd80ca None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:38.418918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:38.424250 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 551/1093] 10.222.0.17 () {56 vars in 1115 bytes} [Sun Aug 7 19:14:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:38.430193 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c9a47-732d-4ac7-b2dc-57723dfd80ca None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:38.445045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 543/1094] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:14:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:14:38.451352 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7bb1f1-6f40-4a6e-b846-32a26aabd654 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:38.544631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c9a47-732d-4ac7-b2dc-57723dfd80ca None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:38.552510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7bb1f1-6f40-4a6e-b846-32a26aabd654 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:38.581485 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aaf6cd-b4ac-49de-8dec-2b8b5b2061b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:38.582120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaf6cd-b4ac-49de-8dec-2b8b5b2061b3 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:38.582397 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaf6cd-b4ac-49de-8dec-2b8b5b2061b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:38.582667 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aaf6cd-b4ac-49de-8dec-2b8b5b2061b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:38.589535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d16312-5447-4ff4-bb05-03d194601fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:38.590163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d16312-5447-4ff4-bb05-03d194601fcd None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:38.590455 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d16312-5447-4ff4-bb05-03d194601fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:38.590727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d16312-5447-4ff4-bb05-03d194601fcd None admin] PATH_INFO: `/v3/projects/f2e6c2ad22784bd7807c964bcda423d1` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:38.591956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d16312-5447-4ff4-bb05-03d194601fcd None admin] RBAC: Authorizing `identity:get_project(project_id=f2e6c2ad22784bd7807c964bcda423d1)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:38.593359 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d16312-5447-4ff4-bb05-03d194601fcd None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:38.593836 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aaf6cd-b4ac-49de-8dec-2b8b5b2061b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:38.595645 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 544/1095] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:14:38 2022] GET /identity/v3/projects/f2e6c2ad22784bd7807c964bcda423d1 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:38.595943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aaf6cd-b4ac-49de-8dec-2b8b5b2061b3 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:38.601227 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d16312-5447-4ff4-bb05-03d194601fcd None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:38.610968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-29aaf6cd-b4ac-49de-8dec-2b8b5b2061b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:38.610968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:38.610968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:38.610968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:38.610968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:38.610968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:38.610968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:38.610968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:38.610968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:38.610968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:38.610968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:38.610968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:38.610968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:38.610968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:38.610968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:38.610968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:38.612321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:38.612321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:38.612321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:38.612321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:38.612321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:38.612321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:38.612321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:38.612321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:38.612321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:38.612321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:38.612321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:38.612321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:38.612321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:38.612321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:38.612321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:38.612321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:38.612321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:38.615775 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:38.615775 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:38.615775 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:38.615775 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 552/1096] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:14:38.618191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aaf6cd-b4ac-49de-8dec-2b8b5b2061b3 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:38.646317 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:38.661554 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:38.726355 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d16312-5447-4ff4-bb05-03d194601fcd None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:38.730436 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aaf6cd-b4ac-49de-8dec-2b8b5b2061b3 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:38.765856 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8989b4-0e7c-4c7b-896d-e77b9979c593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:38.766478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8989b4-0e7c-4c7b-896d-e77b9979c593 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:38.766758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8989b4-0e7c-4c7b-896d-e77b9979c593 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:38.767026 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8989b4-0e7c-4c7b-896d-e77b9979c593 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:38.767757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8989b4-0e7c-4c7b-896d-e77b9979c593 None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=f2e6c2ad22784bd7807c964bcda423d1, user.id=b6f583f1e37242c6a51b12dd4f1ff1e6) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:38.768102 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8989b4-0e7c-4c7b-896d-e77b9979c593 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:38.769531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8989b4-0e7c-4c7b-896d-e77b9979c593 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:38.771017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e541ab-260c-4184-b2b1-15155df3db71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:38.771643 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e541ab-260c-4184-b2b1-15155df3db71 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:38.771924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e541ab-260c-4184-b2b1-15155df3db71 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:38.772278 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e541ab-260c-4184-b2b1-15155df3db71 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:38.772932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e541ab-260c-4184-b2b1-15155df3db71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:38.773232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e541ab-260c-4184-b2b1-15155df3db71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:38.774603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e541ab-260c-4184-b2b1-15155df3db71 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:38.775672 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 545/1097] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:14:38 2022] GET /identity/v3/role_assignments?user.id=b6f583f1e37242c6a51b12dd4f1ff1e6&scope.project.id=f2e6c2ad22784bd7807c964bcda423d1&role.id=4da42f08d107448fb877b42b61bc74ea => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:14:38.787768 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 553/1098] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:38.794384 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8989b4-0e7c-4c7b-896d-e77b9979c593 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:38.912368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8989b4-0e7c-4c7b-896d-e77b9979c593 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:38.922706 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:38.956877 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884b713-1dac-457e-bdb3-a61e78a5418a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:38.957518 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884b713-1dac-457e-bdb3-a61e78a5418a None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:38.960914 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884b713-1dac-457e-bdb3-a61e78a5418a None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:38.960914 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884b713-1dac-457e-bdb3-a61e78a5418a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:38.971613 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884b713-1dac-457e-bdb3-a61e78a5418a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:38.973111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884b713-1dac-457e-bdb3-a61e78a5418a None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:38.985294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-c884b713-1dac-457e-bdb3-a61e78a5418a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:14:38.985294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:38.985294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:38.985294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:38.985294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:38.985294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:38.985294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:38.985294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:38.985294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:38.985294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:38.985294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:38.985294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:38.985294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:14:38.985294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:14:38.985294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:14:38.985294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:38.986800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:38.988148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:38.988148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:38.988148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:14:38.988148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:14:38.988148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:14:38.988148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:14:38.988148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:14:38.988148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:14:38.988148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 19:14:38.988148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:38.988148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 546/1099] 10.222.0.17 () {56 vars in 1093 bytes} [Sun Aug 7 19:14:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:38.992757 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e541ab-260c-4184-b2b1-15155df3db71 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:39.096828 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:39.117133 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e541ab-260c-4184-b2b1-15155df3db71 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:39.153461 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b53b0b-5b35-4f05-ae27-40f5a0bf87aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:39.154721 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b53b0b-5b35-4f05-ae27-40f5a0bf87aa None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:39.154721 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b53b0b-5b35-4f05-ae27-40f5a0bf87aa None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:39.154721 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b53b0b-5b35-4f05-ae27-40f5a0bf87aa None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:39.155669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b53b0b-5b35-4f05-ae27-40f5a0bf87aa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:39.155669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b53b0b-5b35-4f05-ae27-40f5a0bf87aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:39.158229 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b53b0b-5b35-4f05-ae27-40f5a0bf87aa None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:39.179625 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 554/1100] 10.222.0.17 () {56 vars in 1133 bytes} [Sun Aug 7 19:14:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:39.186413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884b713-1dac-457e-bdb3-a61e78a5418a None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:39.296902 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884b713-1dac-457e-bdb3-a61e78a5418a None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:39.347039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fba435-c75b-4a2c-8767-56df8a918a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:39.347670 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fba435-c75b-4a2c-8767-56df8a918a11 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:39.349414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fba435-c75b-4a2c-8767-56df8a918a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:39.349414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fba435-c75b-4a2c-8767-56df8a918a11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:39.393530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:39.431276 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fba435-c75b-4a2c-8767-56df8a918a11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:39.433105 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fba435-c75b-4a2c-8767-56df8a918a11 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:39.539569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 547/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:14:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:14:39.545858 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b53b0b-5b35-4f05-ae27-40f5a0bf87aa None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:39.652204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b53b0b-5b35-4f05-ae27-40f5a0bf87aa None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:39.688228 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177874b5-7f13-4394-b15c-47683d1c4fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:39.688808 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177874b5-7f13-4394-b15c-47683d1c4fac None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:39.689157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177874b5-7f13-4394-b15c-47683d1c4fac None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:39.689386 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177874b5-7f13-4394-b15c-47683d1c4fac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:39.702436 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177874b5-7f13-4394-b15c-47683d1c4fac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:39.703892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177874b5-7f13-4394-b15c-47683d1c4fac None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:39.705931 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:39.723973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-177874b5-7f13-4394-b15c-47683d1c4fac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:39.723973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:39.723973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:39.723973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:39.723973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:39.723973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:39.723973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:39.723973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:39.723973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:39.723973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:39.723973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:39.723973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:39.723973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:14:39.723973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:14:39.723973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:14:39.723973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:14:39.725069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:39.725069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:39.725069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:39.725069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:39.725069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:39.725069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:39.725069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:39.725069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:39.725069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:39.725069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:39.725069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:39.725069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:39.725069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:39.725069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:39.725069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:39.725069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:39.725069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:14:39.726191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:14:39.726191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:14:39.726191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:39.726191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 555/1102] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:39.734116 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fba435-c75b-4a2c-8767-56df8a918a11 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:39.804468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:39.842543 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fba435-c75b-4a2c-8767-56df8a918a11 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:39.878740 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1338e29-7af9-4b40-9ac6-8b20a876f7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:39.879357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1338e29-7af9-4b40-9ac6-8b20a876f7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:39.879639 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1338e29-7af9-4b40-9ac6-8b20a876f7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:39.879911 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1338e29-7af9-4b40-9ac6-8b20a876f7b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:39.880596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1338e29-7af9-4b40-9ac6-8b20a876f7b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:39.880895 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1338e29-7af9-4b40-9ac6-8b20a876f7b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:39.882272 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1338e29-7af9-4b40-9ac6-8b20a876f7b5 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:39.894608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 548/1103] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:39.900367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177874b5-7f13-4394-b15c-47683d1c4fac None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:40.021123 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177874b5-7f13-4394-b15c-47683d1c4fac None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:40.055848 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ff861-9428-4055-952b-ba65ed713e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:40.056508 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ff861-9428-4055-952b-ba65ed713e13 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:40.057119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ff861-9428-4055-952b-ba65ed713e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:40.057419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ff861-9428-4055-952b-ba65ed713e13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:40.064794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ff861-9428-4055-952b-ba65ed713e13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:40.066277 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ff861-9428-4055-952b-ba65ed713e13 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:40.077884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-160ff861-9428-4055-952b-ba65ed713e13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:40.077884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:40.077884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:40.077884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:40.077884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:40.077884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:40.077884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:40.077884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:40.077884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:40.077884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:40.077884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:40.077884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:40.077884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:14:40.077884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:14:40.077884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:14:40.078881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:14:40.078881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:40.078881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:40.078881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:40.078881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:40.078881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:40.078881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:40.078881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:40.078881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:40.078881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:40.078881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:40.078881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:40.078881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:40.078881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:40.078881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:40.078881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:40.078881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:40.079869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:14:40.079869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:14:40.079869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:14:40.079869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:14:40.079869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:14:40.079869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:14:40.079869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:14:40.079869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:14:40.083878 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 556/1104] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:14:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:40.085294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1338e29-7af9-4b40-9ac6-8b20a876f7b5 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:40.191800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1338e29-7af9-4b40-9ac6-8b20a876f7b5 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:40.223193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:40.232142 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853906a-bd38-4917-a033-2da81ac77f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:40.232755 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853906a-bd38-4917-a033-2da81ac77f66 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:40.233029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853906a-bd38-4917-a033-2da81ac77f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:40.233279 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853906a-bd38-4917-a033-2da81ac77f66 None admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:40.233971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853906a-bd38-4917-a033-2da81ac77f66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:40.234275 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853906a-bd38-4917-a033-2da81ac77f66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:40.235707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853906a-bd38-4917-a033-2da81ac77f66 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:40.250251 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 549/1105] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:14:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:40.256386 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ff861-9428-4055-952b-ba65ed713e13 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:40.318237 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:40.368480 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ff861-9428-4055-952b-ba65ed713e13 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:40.406032 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a443d-ca7a-4e73-86d1-ccb50fe2f0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:40.406655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a443d-ca7a-4e73-86d1-ccb50fe2f0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:40.406936 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a443d-ca7a-4e73-86d1-ccb50fe2f0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:40.407211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a443d-ca7a-4e73-86d1-ccb50fe2f0d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:40.407930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a443d-ca7a-4e73-86d1-ccb50fe2f0d2 None admin] RBAC: Adding query filter params (role.id=c5946f310572439687baeed49acd4336, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=a4946aec299f45bd8402cff3159d04d7) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:40.408240 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a443d-ca7a-4e73-86d1-ccb50fe2f0d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:40.409706 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a443d-ca7a-4e73-86d1-ccb50fe2f0d2 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:40.416259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 557/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:14:40 2022] GET /identity/v3/role_assignments?user.id=a4946aec299f45bd8402cff3159d04d7&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=c5946f310572439687baeed49acd4336 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:40.829950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07376ee9-0b83-4e47-b7d7-126d0f630a55 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:40.830495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07376ee9-0b83-4e47-b7d7-126d0f630a55 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:40.830495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07376ee9-0b83-4e47-b7d7-126d0f630a55 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:40.831853 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 550/1107] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:40.837506 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca0c4d-b421-49ca-af13-e8c7d1430bbc None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:40.837967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca0c4d-b421-49ca-af13-e8c7d1430bbc None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:40.838112 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ca0c4d-b421-49ca-af13-e8c7d1430bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:40.857356 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-06ca0c4d-b421-49ca-af13-e8c7d1430bbc None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:40.858551 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-06ca0c4d-b421-49ca-af13-e8c7d1430bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:40.926952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 558/1108] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:40.951768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07376ee9-0b83-4e47-b7d7-126d0f630a55 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:41.072725 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07376ee9-0b83-4e47-b7d7-126d0f630a55 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:41.107120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc8edb-be03-440d-9434-ccfd4adcc1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:41.107709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc8edb-be03-440d-9434-ccfd4adcc1ad None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:41.108098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc8edb-be03-440d-9434-ccfd4adcc1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:41.108297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc8edb-be03-440d-9434-ccfd4adcc1ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:41.109482 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc8edb-be03-440d-9434-ccfd4adcc1ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:41.110928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc8edb-be03-440d-9434-ccfd4adcc1ad None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:41.112583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 551/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:41.118805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca0c4d-b421-49ca-af13-e8c7d1430bbc None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:41.168091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:41.224131 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ca0c4d-b421-49ca-af13-e8c7d1430bbc None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:41.260212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caee8ae-eee0-45d1-b777-f8adad53761b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:41.260926 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caee8ae-eee0-45d1-b777-f8adad53761b None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:41.261205 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caee8ae-eee0-45d1-b777-f8adad53761b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:41.261482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caee8ae-eee0-45d1-b777-f8adad53761b None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:41.262143 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caee8ae-eee0-45d1-b777-f8adad53761b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:41.263617 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caee8ae-eee0-45d1-b777-f8adad53761b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:41.286036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 559/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:14:41 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:14:43.327123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5838140-15ac-444d-adfe-9e34ba8900d0 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:43.327713 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5838140-15ac-444d-adfe-9e34ba8900d0 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:43.327713 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5838140-15ac-444d-adfe-9e34ba8900d0 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:43.328941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 552/1111] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:43.334210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3495a1-b748-4a53-b9d0-90403e75eac2 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:43.334958 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3495a1-b748-4a53-b9d0-90403e75eac2 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:43.335178 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3495a1-b748-4a53-b9d0-90403e75eac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:43.361993 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-0d3495a1-b748-4a53-b9d0-90403e75eac2 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:43.363211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0d3495a1-b748-4a53-b9d0-90403e75eac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:43.431731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 560/1112] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:43.455545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5838140-15ac-444d-adfe-9e34ba8900d0 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:43.518983 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:43.566760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5838140-15ac-444d-adfe-9e34ba8900d0 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:43.601534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0ec03-332e-4a5a-85af-6ae41e6743e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:43.602088 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0ec03-332e-4a5a-85af-6ae41e6743e0 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:43.602360 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0ec03-332e-4a5a-85af-6ae41e6743e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:43.602627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0ec03-332e-4a5a-85af-6ae41e6743e0 None admin] PATH_INFO: `/v3/roles/9a1c954216e34607a6788f44f07a593b` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:43.604035 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0ec03-332e-4a5a-85af-6ae41e6743e0 None admin] RBAC: Authorizing `identity:get_role(role_id=9a1c954216e34607a6788f44f07a593b)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:43.605498 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0ec03-332e-4a5a-85af-6ae41e6743e0 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:43.606808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 553/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:14:43 2022] GET /identity/v3/roles/9a1c954216e34607a6788f44f07a593b => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:43.613163 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3495a1-b748-4a53-b9d0-90403e75eac2 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:43.647352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:43.717643 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3495a1-b748-4a53-b9d0-90403e75eac2 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:43.752559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9817826-289e-4507-9350-79646bf46c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:43.753194 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9817826-289e-4507-9350-79646bf46c6a None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:43.753470 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9817826-289e-4507-9350-79646bf46c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:43.753740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9817826-289e-4507-9350-79646bf46c6a None admin] PATH_INFO: `/v3/users/98a41dad3c0747d1be127477b44a860e` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:43.767493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9817826-289e-4507-9350-79646bf46c6a None admin] RBAC: Authorizing `identity:get_user(user_id=98a41dad3c0747d1be127477b44a860e)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:43.768980 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9817826-289e-4507-9350-79646bf46c6a None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:43.770602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 561/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:14:43 2022] GET /identity/v3/users/98a41dad3c0747d1be127477b44a860e => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:43.776966 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0ec03-332e-4a5a-85af-6ae41e6743e0 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:43.880212 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0ec03-332e-4a5a-85af-6ae41e6743e0 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:43.903341 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:43.916249 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ce224-d64d-481f-a17c-a51e22ef41ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:43.916829 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ce224-d64d-481f-a17c-a51e22ef41ee None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:43.917164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ce224-d64d-481f-a17c-a51e22ef41ee None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:43.917438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ce224-d64d-481f-a17c-a51e22ef41ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:43.987633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ce224-d64d-481f-a17c-a51e22ef41ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:43.989140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ce224-d64d-481f-a17c-a51e22ef41ee None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:44.095226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 554/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:14:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:14:44.101648 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9817826-289e-4507-9350-79646bf46c6a None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:44.197075 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:44.206812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9817826-289e-4507-9350-79646bf46c6a None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:44.242050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755a045e-2631-490a-8d2b-c3a8a64e7d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:44.242942 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a045e-2631-490a-8d2b-c3a8a64e7d73 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:44.243223 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a045e-2631-490a-8d2b-c3a8a64e7d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:44.243494 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755a045e-2631-490a-8d2b-c3a8a64e7d73 None admin] PATH_INFO: `/v3/projects/f2e6c2ad22784bd7807c964bcda423d1` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:44.245276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a045e-2631-490a-8d2b-c3a8a64e7d73 None admin] RBAC: Authorizing `identity:get_project(project_id=f2e6c2ad22784bd7807c964bcda423d1)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:44.247197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755a045e-2631-490a-8d2b-c3a8a64e7d73 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:44.248453 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 562/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:14:44 2022] GET /identity/v3/projects/f2e6c2ad22784bd7807c964bcda423d1 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:44.254921 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ce224-d64d-481f-a17c-a51e22ef41ee None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:44.280229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:44.363814 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ce224-d64d-481f-a17c-a51e22ef41ee None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:44.399982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dab38a-e728-4334-9ad4-bc947777cd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:44.400611 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dab38a-e728-4334-9ad4-bc947777cd10 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:44.400890 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dab38a-e728-4334-9ad4-bc947777cd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:44.401162 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dab38a-e728-4334-9ad4-bc947777cd10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:44.401940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dab38a-e728-4334-9ad4-bc947777cd10 None admin] RBAC: Adding query filter params (role.id=9a1c954216e34607a6788f44f07a593b, scope.project.id=f2e6c2ad22784bd7807c964bcda423d1, user.id=98a41dad3c0747d1be127477b44a860e) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:44.402246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dab38a-e728-4334-9ad4-bc947777cd10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:44.403672 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dab38a-e728-4334-9ad4-bc947777cd10 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:44.409158 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 555/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:14:44 2022] GET /identity/v3/role_assignments?user.id=98a41dad3c0747d1be127477b44a860e&scope.project.id=f2e6c2ad22784bd7807c964bcda423d1&role.id=9a1c954216e34607a6788f44f07a593b => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:46.364582 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa46b7-47a8-4ebd-a860-d9ece7405c39 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:46.365108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa46b7-47a8-4ebd-a860-d9ece7405c39 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:46.365108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa46b7-47a8-4ebd-a860-d9ece7405c39 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:46.366376 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 563/1118] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:46.372270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062e78f-7eac-45de-b501-48518e50739a None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:46.372681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062e78f-7eac-45de-b501-48518e50739a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:46.372810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062e78f-7eac-45de-b501-48518e50739a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:46.391125 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-e062e78f-7eac-45de-b501-48518e50739a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:46.392343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e062e78f-7eac-45de-b501-48518e50739a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:46.461960 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 556/1119] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:46.484511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa46b7-47a8-4ebd-a860-d9ece7405c39 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:46.590007 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa46b7-47a8-4ebd-a860-d9ece7405c39 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:46.624745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eca6c4-5608-4078-aaec-6d7780395a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:46.625284 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eca6c4-5608-4078-aaec-6d7780395a2f None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:46.625557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eca6c4-5608-4078-aaec-6d7780395a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:46.625824 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eca6c4-5608-4078-aaec-6d7780395a2f None admin] PATH_INFO: `/v3/roles/9a1c954216e34607a6788f44f07a593b` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:46.627230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eca6c4-5608-4078-aaec-6d7780395a2f None admin] RBAC: Authorizing `identity:get_role(role_id=9a1c954216e34607a6788f44f07a593b)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:46.628641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eca6c4-5608-4078-aaec-6d7780395a2f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:46.629954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 564/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:14:46 2022] GET /identity/v3/roles/9a1c954216e34607a6788f44f07a593b => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:46.636118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062e78f-7eac-45de-b501-48518e50739a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:46.654604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:46.739432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062e78f-7eac-45de-b501-48518e50739a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:46.775652 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade07036-67ca-4330-8389-b2ed5d827443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:46.776405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade07036-67ca-4330-8389-b2ed5d827443 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:46.776687 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade07036-67ca-4330-8389-b2ed5d827443 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:46.776971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade07036-67ca-4330-8389-b2ed5d827443 None admin] PATH_INFO: `/v3/users/98a41dad3c0747d1be127477b44a860e` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:46.778112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade07036-67ca-4330-8389-b2ed5d827443 None admin] RBAC: Authorizing `identity:get_user(user_id=98a41dad3c0747d1be127477b44a860e)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:46.779550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade07036-67ca-4330-8389-b2ed5d827443 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:46.781145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 557/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:14:46 2022] GET /identity/v3/users/98a41dad3c0747d1be127477b44a860e => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:46.787240 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eca6c4-5608-4078-aaec-6d7780395a2f None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:46.817229 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:46.891541 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eca6c4-5608-4078-aaec-6d7780395a2f None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:46.926402 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df38b7c-556f-4b46-8a32-3b9016d33425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:46.927031 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df38b7c-556f-4b46-8a32-3b9016d33425 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:46.927308 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df38b7c-556f-4b46-8a32-3b9016d33425 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:46.927576 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df38b7c-556f-4b46-8a32-3b9016d33425 None admin] PATH_INFO: `/v3/projects/f2e6c2ad22784bd7807c964bcda423d1` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:46.928733 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df38b7c-556f-4b46-8a32-3b9016d33425 None admin] RBAC: Authorizing `identity:get_project(project_id=f2e6c2ad22784bd7807c964bcda423d1)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:46.930170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df38b7c-556f-4b46-8a32-3b9016d33425 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:46.932178 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 565/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:14:46 2022] GET /identity/v3/projects/f2e6c2ad22784bd7807c964bcda423d1 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:46.938587 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade07036-67ca-4330-8389-b2ed5d827443 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:47.029126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:47.045321 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade07036-67ca-4330-8389-b2ed5d827443 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:47.079204 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd22e88-ff3c-4a79-a3bc-fc8db16e179c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:47.079859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd22e88-ff3c-4a79-a3bc-fc8db16e179c None admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:47.080172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd22e88-ff3c-4a79-a3bc-fc8db16e179c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:47.080451 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd22e88-ff3c-4a79-a3bc-fc8db16e179c None admin] PATH_INFO: `/v3/projects/f2e6c2ad22784bd7807c964bcda423d1/users/98a41dad3c0747d1be127477b44a860e/roles/9a1c954216e34607a6788f44f07a593b` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:47.082632 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd22e88-ff3c-4a79-a3bc-fc8db16e179c None admin] RBAC: Authorizing `identity:create_grant(project_id=f2e6c2ad22784bd7807c964bcda423d1, user_id=98a41dad3c0747d1be127477b44a860e, role_id=9a1c954216e34607a6788f44f07a593b)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:47.084101 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd22e88-ff3c-4a79-a3bc-fc8db16e179c None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:47.097185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 558/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:14:46 2022] PUT /identity/v3/projects/f2e6c2ad22784bd7807c964bcda423d1/users/98a41dad3c0747d1be127477b44a860e/roles/9a1c954216e34607a6788f44f07a593b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:14:49.060294 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d288fcb-bd6a-491a-9409-60fc65d60541 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:49.060839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d288fcb-bd6a-491a-9409-60fc65d60541 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:49.060839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d288fcb-bd6a-491a-9409-60fc65d60541 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:49.062162 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 566/1124] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:49.067713 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb273a73-f95d-4196-84cf-8b323dd729c3 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:49.068182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb273a73-f95d-4196-84cf-8b323dd729c3 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:49.068273 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb273a73-f95d-4196-84cf-8b323dd729c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:49.086710 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-eb273a73-f95d-4196-84cf-8b323dd729c3 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:49.087889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-eb273a73-f95d-4196-84cf-8b323dd729c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:49.155387 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 559/1125] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 19:14:49.179288 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d288fcb-bd6a-491a-9409-60fc65d60541 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:49.280479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:49.288155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d288fcb-bd6a-491a-9409-60fc65d60541 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:49.323950 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c561f5-57b5-4c1f-a418-958204b964de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:49.324610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c561f5-57b5-4c1f-a418-958204b964de None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:49.324885 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c561f5-57b5-4c1f-a418-958204b964de None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:49.325158 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c561f5-57b5-4c1f-a418-958204b964de None admin] PATH_INFO: `/v3/roles/9a1c954216e34607a6788f44f07a593b` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:49.326592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c561f5-57b5-4c1f-a418-958204b964de None admin] RBAC: Authorizing `identity:get_role(role_id=9a1c954216e34607a6788f44f07a593b)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:49.327975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c561f5-57b5-4c1f-a418-958204b964de None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:49.329348 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 567/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:14:49 2022] GET /identity/v3/roles/9a1c954216e34607a6788f44f07a593b => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:49.335552 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb273a73-f95d-4196-84cf-8b323dd729c3 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:49.412587 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:49.438392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb273a73-f95d-4196-84cf-8b323dd729c3 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:49.472632 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acb69e-339d-4b5c-85b9-10621a10580a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:49.473261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acb69e-339d-4b5c-85b9-10621a10580a None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:49.473534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acb69e-339d-4b5c-85b9-10621a10580a None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:49.473802 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acb69e-339d-4b5c-85b9-10621a10580a None admin] PATH_INFO: `/v3/users/98a41dad3c0747d1be127477b44a860e` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:49.474907 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7acb69e-339d-4b5c-85b9-10621a10580a None admin] RBAC: Authorizing `identity:get_user(user_id=98a41dad3c0747d1be127477b44a860e)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:49.476356 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7acb69e-339d-4b5c-85b9-10621a10580a None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:49.477927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 560/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:14:49 2022] GET /identity/v3/users/98a41dad3c0747d1be127477b44a860e => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:49.484159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c561f5-57b5-4c1f-a418-958204b964de None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:49.589967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c561f5-57b5-4c1f-a418-958204b964de None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:49.624510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e5d4d-669b-48e0-9cf6-94ae3275d73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:49.625123 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e5d4d-669b-48e0-9cf6-94ae3275d73e None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:49.625396 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e5d4d-669b-48e0-9cf6-94ae3275d73e None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:49.625671 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18e5d4d-669b-48e0-9cf6-94ae3275d73e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:49.658841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:49.699755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e5d4d-669b-48e0-9cf6-94ae3275d73e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:49.701265 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18e5d4d-669b-48e0-9cf6-94ae3275d73e None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:49.801544 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 568/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:14:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:14:49.808406 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acb69e-339d-4b5c-85b9-10621a10580a None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:49.907047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acb69e-339d-4b5c-85b9-10621a10580a None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:49.941519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a6d01-b246-4b67-b83d-b61f4d389c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:49.942150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a6d01-b246-4b67-b83d-b61f4d389c00 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:49.942425 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a6d01-b246-4b67-b83d-b61f4d389c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:49.942694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a6d01-b246-4b67-b83d-b61f4d389c00 None admin] PATH_INFO: `/v3/projects/f2e6c2ad22784bd7807c964bcda423d1` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:49.943824 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606a6d01-b246-4b67-b83d-b61f4d389c00 None admin] RBAC: Authorizing `identity:get_project(project_id=f2e6c2ad22784bd7807c964bcda423d1)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:49.945297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606a6d01-b246-4b67-b83d-b61f4d389c00 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:49.947039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 561/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:14:49 2022] GET /identity/v3/projects/f2e6c2ad22784bd7807c964bcda423d1 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:49.953554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e5d4d-669b-48e0-9cf6-94ae3275d73e None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:50.023176 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:50.062501 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18e5d4d-669b-48e0-9cf6-94ae3275d73e None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:50.097656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff98ca-2e5f-4437-a78f-5bb55d3035b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:50.098253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff98ca-2e5f-4437-a78f-5bb55d3035b5 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:50.098522 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff98ca-2e5f-4437-a78f-5bb55d3035b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:50.098786 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff98ca-2e5f-4437-a78f-5bb55d3035b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:50.099503 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff98ca-2e5f-4437-a78f-5bb55d3035b5 None admin] RBAC: Adding query filter params (role.id=9a1c954216e34607a6788f44f07a593b, scope.project.id=f2e6c2ad22784bd7807c964bcda423d1, user.id=98a41dad3c0747d1be127477b44a860e) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:50.099800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff98ca-2e5f-4437-a78f-5bb55d3035b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:50.101249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff98ca-2e5f-4437-a78f-5bb55d3035b5 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:50.107099 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 569/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:14:49 2022] GET /identity/v3/role_assignments?user.id=98a41dad3c0747d1be127477b44a860e&scope.project.id=f2e6c2ad22784bd7807c964bcda423d1&role.id=9a1c954216e34607a6788f44f07a593b => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:52.088244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bae9d-b22b-48a9-b357-016a45991f5d None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:52.088823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bae9d-b22b-48a9-b357-016a45991f5d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:52.088823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bae9d-b22b-48a9-b357-016a45991f5d None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:52.090223 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 562/1131] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:52.095732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209bf9c-16d4-478c-ae27-9e42f856e960 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:52.096184 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209bf9c-16d4-478c-ae27-9e42f856e960 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:52.096290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209bf9c-16d4-478c-ae27-9e42f856e960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:52.114615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-8209bf9c-16d4-478c-ae27-9e42f856e960 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:52.115800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8209bf9c-16d4-478c-ae27-9e42f856e960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:52.185409 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 570/1132] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:52.208798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bae9d-b22b-48a9-b357-016a45991f5d None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:52.225015 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:52.318609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bae9d-b22b-48a9-b357-016a45991f5d None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:52.353321 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e307fc-d067-494a-87e9-92fa629bead6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:52.353954 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e307fc-d067-494a-87e9-92fa629bead6 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:52.354232 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e307fc-d067-494a-87e9-92fa629bead6 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:52.354564 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e307fc-d067-494a-87e9-92fa629bead6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:52.355729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e307fc-d067-494a-87e9-92fa629bead6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:52.357198 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e307fc-d067-494a-87e9-92fa629bead6 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:52.358802 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 563/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:52.365036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209bf9c-16d4-478c-ae27-9e42f856e960 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:52.422636 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:52.467895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209bf9c-16d4-478c-ae27-9e42f856e960 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:52.502899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1d030-780a-45f0-8f8a-c18691764b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:52.503494 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1d030-780a-45f0-8f8a-c18691764b2f None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:52.503778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1d030-780a-45f0-8f8a-c18691764b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:52.504040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1d030-780a-45f0-8f8a-c18691764b2f None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:52.504690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1d030-780a-45f0-8f8a-c18691764b2f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:52.506126 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1d030-780a-45f0-8f8a-c18691764b2f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:52.540044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 571/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Aug 7 19:14:52 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:14:54.545087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd70bd4-35e7-4d42-9883-0e6d6a040fd8 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:54.545681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd70bd4-35e7-4d42-9883-0e6d6a040fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:54.545681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd70bd4-35e7-4d42-9883-0e6d6a040fd8 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:54.547010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 564/1135] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:54.552672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634a6a5-a5f3-49c6-8008-82d690f6da44 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:54.553183 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634a6a5-a5f3-49c6-8008-82d690f6da44 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:54.553443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634a6a5-a5f3-49c6-8008-82d690f6da44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:54.577248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-0634a6a5-a5f3-49c6-8008-82d690f6da44 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:54.578487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0634a6a5-a5f3-49c6-8008-82d690f6da44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:54.655783 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 572/1136] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:54.680253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd70bd4-35e7-4d42-9883-0e6d6a040fd8 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:54.767909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:54.789100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd70bd4-35e7-4d42-9883-0e6d6a040fd8 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:54.824436 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d254a-d922-4408-977f-7043bc52c186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:54.824954 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d254a-d922-4408-977f-7043bc52c186 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:54.825236 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d254a-d922-4408-977f-7043bc52c186 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:54.825500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d254a-d922-4408-977f-7043bc52c186 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:54.826663 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d254a-d922-4408-977f-7043bc52c186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:54.828132 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d254a-d922-4408-977f-7043bc52c186 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:54.829707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 565/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:14:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:54.835743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634a6a5-a5f3-49c6-8008-82d690f6da44 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:54.879920 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:54.949810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634a6a5-a5f3-49c6-8008-82d690f6da44 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:54.999310 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f98591-74aa-4ea0-90ac-dc5f14c0196c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:54.999920 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f98591-74aa-4ea0-90ac-dc5f14c0196c None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:55.000226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f98591-74aa-4ea0-90ac-dc5f14c0196c None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:55.000497 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f98591-74aa-4ea0-90ac-dc5f14c0196c None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:55.001182 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f98591-74aa-4ea0-90ac-dc5f14c0196c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:55.002642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f98591-74aa-4ea0-90ac-dc5f14c0196c None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:55.029819 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 573/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:14:54 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:14:57.100849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb356d-ba59-46d0-b91e-779f198bbabb None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:57.101644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb356d-ba59-46d0-b91e-779f198bbabb None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:57.101644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbb356d-ba59-46d0-b91e-779f198bbabb None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:57.102736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 566/1139] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:14:57.108892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd04e15-97b3-4c13-9f1b-09f4c6334597 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:57.109368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd04e15-97b3-4c13-9f1b-09f4c6334597 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:57.109456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd04e15-97b3-4c13-9f1b-09f4c6334597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:57.128931 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-8cd04e15-97b3-4c13-9f1b-09f4c6334597 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:14:57.130143 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8cd04e15-97b3-4c13-9f1b-09f4c6334597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:14:57.200791 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 574/1140] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:14:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:14:57.224288 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb356d-ba59-46d0-b91e-779f198bbabb None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:57.333179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbb356d-ba59-46d0-b91e-779f198bbabb None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:57.367294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1e870-771d-4042-ab77-2676c53cdcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:57.367929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1e870-771d-4042-ab77-2676c53cdcc7 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:57.368231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1e870-771d-4042-ab77-2676c53cdcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:57.368535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1e870-771d-4042-ab77-2676c53cdcc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:57.379940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1e870-771d-4042-ab77-2676c53cdcc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:14:57.393796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-e9f1e870-771d-4042-ab77-2676c53cdcc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:14:57.393796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:14:57.393796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:14:57.393796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:14:57.393796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:14:57.393796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:14:57.393796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:14:57.393796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:14:57.393796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:14:57.393796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:14:57.393796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:14:57.393796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:14:57.393796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:14:57.393796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:14:57.393796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:14:57.393796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:14:57.395028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:14:57.395028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:14:57.395028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:14:57.395028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:14:57.395028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:14:57.395028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:14:57.395028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:14:57.395028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:14:57.395028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:14:57.395028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:14:57.395028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:14:57.395028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:14:57.395028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:14:57.395028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:14:57.395028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:14:57.395028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:14:57.396401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:14:57.396401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:14:57.396401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:14:57.396401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:14:57.396401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:14:57.396401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:14:57.396401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:14:57.396401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:14:57.396401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:14:57.396401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:14:57.396401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:14:57.396401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:14:57.396401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 567/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:14:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:14:57.401481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd04e15-97b3-4c13-9f1b-09f4c6334597 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:57.429692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:57.508690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd04e15-97b3-4c13-9f1b-09f4c6334597 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:57.545007 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04da722-fbb2-4415-8af0-0c1299f9a076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:57.545616 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04da722-fbb2-4415-8af0-0c1299f9a076 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:57.545894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04da722-fbb2-4415-8af0-0c1299f9a076 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:57.546166 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04da722-fbb2-4415-8af0-0c1299f9a076 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:57.546834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04da722-fbb2-4415-8af0-0c1299f9a076 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:57.547143 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04da722-fbb2-4415-8af0-0c1299f9a076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:57.548540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04da722-fbb2-4415-8af0-0c1299f9a076 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:57.557709 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 575/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:14:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:57.563833 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1e870-771d-4042-ab77-2676c53cdcc7 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:57.577192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:57.677957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1e870-771d-4042-ab77-2676c53cdcc7 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:57.714947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3794610-4ecc-466f-b563-b68e9dc30ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:57.715586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3794610-4ecc-466f-b563-b68e9dc30ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:57.715864 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3794610-4ecc-466f-b563-b68e9dc30ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:57.716174 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3794610-4ecc-466f-b563-b68e9dc30ba5 None admin] PATH_INFO: `/v3/users/0749084027d64784ac7253438c7a536e` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:57.730509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3794610-4ecc-466f-b563-b68e9dc30ba5 None admin] RBAC: Authorizing `identity:get_user(user_id=0749084027d64784ac7253438c7a536e)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:57.731969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3794610-4ecc-466f-b563-b68e9dc30ba5 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:57.733655 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 568/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:14:57 2022] GET /identity/v3/users/0749084027d64784ac7253438c7a536e => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:57.745273 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04da722-fbb2-4415-8af0-0c1299f9a076 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:57.841423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:57.854809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04da722-fbb2-4415-8af0-0c1299f9a076 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:57.891913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2830b-02e0-4f94-b987-46fbecbd48bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:57.892579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2830b-02e0-4f94-b987-46fbecbd48bd None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:57.892860 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2830b-02e0-4f94-b987-46fbecbd48bd None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:57.893132 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2830b-02e0-4f94-b987-46fbecbd48bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:57.965970 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e2830b-02e0-4f94-b987-46fbecbd48bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:57.967346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e2830b-02e0-4f94-b987-46fbecbd48bd None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:58.068710 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:58.084090 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 576/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:14:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:14:58.090480 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3794610-4ecc-466f-b563-b68e9dc30ba5 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:58.162687 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:14:58.202970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3794610-4ecc-466f-b563-b68e9dc30ba5 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:58.240151 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753179ab-5167-4533-97a7-e8ae748e4e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:58.240711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753179ab-5167-4533-97a7-e8ae748e4e55 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:58.240994 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753179ab-5167-4533-97a7-e8ae748e4e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:58.241265 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753179ab-5167-4533-97a7-e8ae748e4e55 None admin] PATH_INFO: `/v3/projects/6c56ab08e1f9402da58583d702265927` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:58.242443 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753179ab-5167-4533-97a7-e8ae748e4e55 None admin] RBAC: Authorizing `identity:get_project(project_id=6c56ab08e1f9402da58583d702265927)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:58.243918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753179ab-5167-4533-97a7-e8ae748e4e55 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:58.245616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 569/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:14:58 2022] GET /identity/v3/projects/6c56ab08e1f9402da58583d702265927 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:14:58.252082 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2830b-02e0-4f94-b987-46fbecbd48bd None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:14:58.358584 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2830b-02e0-4f94-b987-46fbecbd48bd None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:14:58.394577 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c725b4-9ac6-4944-aa69-f5ae369893bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:14:58.395176 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c725b4-9ac6-4944-aa69-f5ae369893bc None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:14:58.395448 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c725b4-9ac6-4944-aa69-f5ae369893bc None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:14:58.395728 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c725b4-9ac6-4944-aa69-f5ae369893bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:14:58.396490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c725b4-9ac6-4944-aa69-f5ae369893bc None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=6c56ab08e1f9402da58583d702265927, user.id=0749084027d64784ac7253438c7a536e) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:14:58.396797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c725b4-9ac6-4944-aa69-f5ae369893bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:14:58.398246 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c725b4-9ac6-4944-aa69-f5ae369893bc None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:14:58.404014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 577/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:14:58 2022] GET /identity/v3/role_assignments?user.id=0749084027d64784ac7253438c7a536e&scope.project.id=6c56ab08e1f9402da58583d702265927&role.id=4da42f08d107448fb877b42b61bc74ea => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:00.410002 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3180f-80d1-4e6e-a689-b3044253cec6 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:00.410534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3180f-80d1-4e6e-a689-b3044253cec6 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:00.410534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd3180f-80d1-4e6e-a689-b3044253cec6 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:00.411798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 570/1147] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:15:00.417973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83391f67-8010-4a71-8f72-660abe1b872f None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:00.418454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83391f67-8010-4a71-8f72-660abe1b872f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:00.418454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83391f67-8010-4a71-8f72-660abe1b872f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:00.436835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-83391f67-8010-4a71-8f72-660abe1b872f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:15:00.438014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-83391f67-8010-4a71-8f72-660abe1b872f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:15:00.489585 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:00.511563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 578/1148] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:15:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:15:00.534062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3180f-80d1-4e6e-a689-b3044253cec6 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:00.638430 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd3180f-80d1-4e6e-a689-b3044253cec6 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:00.674538 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db695d68-9cd0-4cb9-b513-058298ed6a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:00.675170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db695d68-9cd0-4cb9-b513-058298ed6a65 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:00.675477 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db695d68-9cd0-4cb9-b513-058298ed6a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:00.675751 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db695d68-9cd0-4cb9-b513-058298ed6a65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:00.677699 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:00.687489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db695d68-9cd0-4cb9-b513-058298ed6a65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:15:00.692200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-db695d68-9cd0-4cb9-b513-058298ed6a65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:15:00.692200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:15:00.692200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:15:00.692200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:15:00.692200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:15:00.692200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:15:00.692200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:15:00.692200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:15:00.692200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:15:00.692200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:15:00.692200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:15:00.692200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:15:00.692200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:15:00.692200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:15:00.692200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:15:00.692200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:15:00.693526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:15:00.693526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:15:00.693526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:15:00.693526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:15:00.693526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:15:00.693526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:15:00.693526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:15:00.693526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:15:00.693526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:15:00.693526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:15:00.693526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:15:00.693526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:15:00.693526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:15:00.693526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:15:00.693526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:15:00.693526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:15:00.694694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:15:00.694694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:15:00.694694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:15:00.694694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:15:00.694694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:15:00.694694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:15:00.694694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:15:00.694694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:15:00.694694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:15:00.694694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:15:00.694694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:15:00.694694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:15:00.694694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 571/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:15:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:15:00.699751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83391f67-8010-4a71-8f72-660abe1b872f None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:00.800520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83391f67-8010-4a71-8f72-660abe1b872f None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:00.835410 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2205b5b1-bbfa-43a4-a8f9-a5a40683c0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:00.836034 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205b5b1-bbfa-43a4-a8f9-a5a40683c0ef None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:00.836307 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205b5b1-bbfa-43a4-a8f9-a5a40683c0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:00.836578 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205b5b1-bbfa-43a4-a8f9-a5a40683c0ef None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:00.837233 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2205b5b1-bbfa-43a4-a8f9-a5a40683c0ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:15:00.837532 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2205b5b1-bbfa-43a4-a8f9-a5a40683c0ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:00.838972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2205b5b1-bbfa-43a4-a8f9-a5a40683c0ef None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:00.847059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 579/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:15:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:00.853335 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db695d68-9cd0-4cb9-b513-058298ed6a65 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:00.959162 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db695d68-9cd0-4cb9-b513-058298ed6a65 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:00.993913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44d83d-6481-42a9-a3a1-02d21f2cf3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:00.994530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44d83d-6481-42a9-a3a1-02d21f2cf3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:00.994804 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44d83d-6481-42a9-a3a1-02d21f2cf3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:00.995110 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44d83d-6481-42a9-a3a1-02d21f2cf3f2 None admin] PATH_INFO: `/v3/users/0749084027d64784ac7253438c7a536e` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:00.996283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44d83d-6481-42a9-a3a1-02d21f2cf3f2 None admin] RBAC: Authorizing `identity:get_user(user_id=0749084027d64784ac7253438c7a536e)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:00.997709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44d83d-6481-42a9-a3a1-02d21f2cf3f2 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:00.999264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 572/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:15:00 2022] GET /identity/v3/users/0749084027d64784ac7253438c7a536e => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:01.005456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2205b5b1-bbfa-43a4-a8f9-a5a40683c0ef None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:01.054418 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:01.112543 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2205b5b1-bbfa-43a4-a8f9-a5a40683c0ef None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:01.148426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2120b377-7977-4f25-954d-dcc70803ec13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:01.149069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120b377-7977-4f25-954d-dcc70803ec13 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:01.149346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120b377-7977-4f25-954d-dcc70803ec13 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:01.149626 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2120b377-7977-4f25-954d-dcc70803ec13 None admin] PATH_INFO: `/v3/projects/6c56ab08e1f9402da58583d702265927` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:01.150801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120b377-7977-4f25-954d-dcc70803ec13 None admin] RBAC: Authorizing `identity:get_project(project_id=6c56ab08e1f9402da58583d702265927)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:01.152392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2120b377-7977-4f25-954d-dcc70803ec13 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:01.154061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 580/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:15:01 2022] GET /identity/v3/projects/6c56ab08e1f9402da58583d702265927 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:01.160427 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44d83d-6481-42a9-a3a1-02d21f2cf3f2 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:01.222574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:01.262581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44d83d-6481-42a9-a3a1-02d21f2cf3f2 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:01.294656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca9cb2-603c-489f-800e-56e11b11b61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:01.295304 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca9cb2-603c-489f-800e-56e11b11b61c None admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:01.295587 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca9cb2-603c-489f-800e-56e11b11b61c None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:01.295863 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca9cb2-603c-489f-800e-56e11b11b61c None admin] PATH_INFO: `/v3/projects/6c56ab08e1f9402da58583d702265927/users/0749084027d64784ac7253438c7a536e/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:01.297891 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca9cb2-603c-489f-800e-56e11b11b61c None admin] RBAC: Authorizing `identity:create_grant(project_id=6c56ab08e1f9402da58583d702265927, user_id=0749084027d64784ac7253438c7a536e, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:01.299327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca9cb2-603c-489f-800e-56e11b11b61c None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:01.314467 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 573/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:15:01 2022] PUT /identity/v3/projects/6c56ab08e1f9402da58583d702265927/users/0749084027d64784ac7253438c7a536e/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:15:03.277133 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cabb80-8da7-4dfc-9da0-f7e7706fad51 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:03.277660 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cabb80-8da7-4dfc-9da0-f7e7706fad51 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:03.277660 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cabb80-8da7-4dfc-9da0-f7e7706fad51 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:03.278978 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 581/1154] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:15:03.284331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe717b0-2212-442d-9530-ec5e2b6bf5a8 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:03.284777 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe717b0-2212-442d-9530-ec5e2b6bf5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:03.284872 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe717b0-2212-442d-9530-ec5e2b6bf5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:03.303243 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-8fe717b0-2212-442d-9530-ec5e2b6bf5a8 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:15:03.304494 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8fe717b0-2212-442d-9530-ec5e2b6bf5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:15:03.372526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 574/1155] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:15:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:15:03.395824 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cabb80-8da7-4dfc-9da0-f7e7706fad51 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:03.500526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cabb80-8da7-4dfc-9da0-f7e7706fad51 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:03.513697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:03.538171 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628d317-08d7-410a-bab3-fbd62006a9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:03.538738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628d317-08d7-410a-bab3-fbd62006a9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:03.539002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628d317-08d7-410a-bab3-fbd62006a9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:03.539277 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628d317-08d7-410a-bab3-fbd62006a9c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:03.550119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5628d317-08d7-410a-bab3-fbd62006a9c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:15:03.554810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-5628d317-08d7-410a-bab3-fbd62006a9c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:15:03.554810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:15:03.554810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:15:03.554810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:15:03.554810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:15:03.554810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:15:03.554810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:15:03.554810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:15:03.554810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:15:03.554810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:15:03.554810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:15:03.554810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:15:03.554810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:15:03.554810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:15:03.554810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:15:03.554810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:15:03.555919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:15:03.555919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:15:03.555919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:15:03.555919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:15:03.555919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:15:03.555919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:15:03.555919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:15:03.555919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:15:03.555919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:15:03.555919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:15:03.555919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:15:03.555919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:15:03.555919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:15:03.555919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:15:03.555919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:15:03.555919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:15:03.557620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:15:03.557620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:15:03.557620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:15:03.557620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:15:03.557620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:15:03.557620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:15:03.557620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:15:03.557620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:15:03.557620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:15:03.557620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:15:03.557620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:15:03.557620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:15:03.557620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 582/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:15:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:15:03.562757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe717b0-2212-442d-9530-ec5e2b6bf5a8 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:03.616368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:03.664009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe717b0-2212-442d-9530-ec5e2b6bf5a8 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:03.699028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b56dc-3db0-4234-95e4-c921f2884f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:03.699684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b56dc-3db0-4234-95e4-c921f2884f4e None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:03.699963 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b56dc-3db0-4234-95e4-c921f2884f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:03.700262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b56dc-3db0-4234-95e4-c921f2884f4e None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:03.700927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b56dc-3db0-4234-95e4-c921f2884f4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:15:03.701229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b56dc-3db0-4234-95e4-c921f2884f4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:03.702595 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b56dc-3db0-4234-95e4-c921f2884f4e None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:03.710683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 575/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:15:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:03.716837 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628d317-08d7-410a-bab3-fbd62006a9c7 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:03.820888 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628d317-08d7-410a-bab3-fbd62006a9c7 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:03.855594 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1d395-8b7e-4079-bc1d-dcd5f5dfdc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:03.856246 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1d395-8b7e-4079-bc1d-dcd5f5dfdc26 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:03.856534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1d395-8b7e-4079-bc1d-dcd5f5dfdc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:03.856804 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1d395-8b7e-4079-bc1d-dcd5f5dfdc26 None admin] PATH_INFO: `/v3/users/0749084027d64784ac7253438c7a536e` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:03.857978 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1d395-8b7e-4079-bc1d-dcd5f5dfdc26 None admin] RBAC: Authorizing `identity:get_user(user_id=0749084027d64784ac7253438c7a536e)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:03.859425 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1d395-8b7e-4079-bc1d-dcd5f5dfdc26 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:03.861049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 583/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:15:03 2022] GET /identity/v3/users/0749084027d64784ac7253438c7a536e => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:03.867123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b56dc-3db0-4234-95e4-c921f2884f4e None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:03.970508 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b56dc-3db0-4234-95e4-c921f2884f4e None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:03.984195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:04.005165 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e255b88-2d98-4b1d-87c8-5c954b8dafdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:04.005863 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e255b88-2d98-4b1d-87c8-5c954b8dafdc None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:04.006017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e255b88-2d98-4b1d-87c8-5c954b8dafdc None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:04.006314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e255b88-2d98-4b1d-87c8-5c954b8dafdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:04.076820 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e255b88-2d98-4b1d-87c8-5c954b8dafdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:04.078207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e255b88-2d98-4b1d-87c8-5c954b8dafdc None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:04.180577 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 576/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:15:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:15:04.187311 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1d395-8b7e-4079-bc1d-dcd5f5dfdc26 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:04.229173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:04.286094 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1d395-8b7e-4079-bc1d-dcd5f5dfdc26 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:04.320656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad59641-0566-4d53-a1a1-c3c6a0c57f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:04.321276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad59641-0566-4d53-a1a1-c3c6a0c57f39 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:04.321556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad59641-0566-4d53-a1a1-c3c6a0c57f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:04.321825 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad59641-0566-4d53-a1a1-c3c6a0c57f39 None admin] PATH_INFO: `/v3/projects/6c56ab08e1f9402da58583d702265927` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:04.322956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad59641-0566-4d53-a1a1-c3c6a0c57f39 None admin] RBAC: Authorizing `identity:get_project(project_id=6c56ab08e1f9402da58583d702265927)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:04.324479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad59641-0566-4d53-a1a1-c3c6a0c57f39 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:04.326114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 584/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:15:04 2022] GET /identity/v3/projects/6c56ab08e1f9402da58583d702265927 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:04.332800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e255b88-2d98-4b1d-87c8-5c954b8dafdc None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:04.348895 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:04.437664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e255b88-2d98-4b1d-87c8-5c954b8dafdc None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:04.472294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc636d61-6f73-4fc6-9482-b871bdab8ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:04.472898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc636d61-6f73-4fc6-9482-b871bdab8ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:04.473179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc636d61-6f73-4fc6-9482-b871bdab8ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:04.473446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc636d61-6f73-4fc6-9482-b871bdab8ce7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:04.474164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc636d61-6f73-4fc6-9482-b871bdab8ce7 None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=6c56ab08e1f9402da58583d702265927, user.id=0749084027d64784ac7253438c7a536e) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:15:04.474470 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc636d61-6f73-4fc6-9482-b871bdab8ce7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:04.475890 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc636d61-6f73-4fc6-9482-b871bdab8ce7 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:04.481907 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 577/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:15:04 2022] GET /identity/v3/role_assignments?user.id=0749084027d64784ac7253438c7a536e&scope.project.id=6c56ab08e1f9402da58583d702265927&role.id=4da42f08d107448fb877b42b61bc74ea => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:06.490483 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808a611-f2b3-48f3-94a7-7bf0964bbb43 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:06.491062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808a611-f2b3-48f3-94a7-7bf0964bbb43 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:06.491062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808a611-f2b3-48f3-94a7-7bf0964bbb43 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:06.492429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 585/1162] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:15:06.497785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6712c-6f7f-43d0-b48b-fef29a0ae4a0 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:06.498242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6712c-6f7f-43d0-b48b-fef29a0ae4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:06.498332 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6712c-6f7f-43d0-b48b-fef29a0ae4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:06.517779 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-8ee6712c-6f7f-43d0-b48b-fef29a0ae4a0 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:15:06.518978 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8ee6712c-6f7f-43d0-b48b-fef29a0ae4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:15:06.590436 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 578/1163] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:15:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:15:06.614769 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808a611-f2b3-48f3-94a7-7bf0964bbb43 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:06.725702 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808a611-f2b3-48f3-94a7-7bf0964bbb43 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:06.737417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:06.765179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0feb3-707e-440b-82d4-cf1c0ae4868b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:06.765716 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0feb3-707e-440b-82d4-cf1c0ae4868b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:06.765995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0feb3-707e-440b-82d4-cf1c0ae4868b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:06.766271 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0feb3-707e-440b-82d4-cf1c0ae4868b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:06.837312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0feb3-707e-440b-82d4-cf1c0ae4868b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:06.838829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0feb3-707e-440b-82d4-cf1c0ae4868b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:06.943509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 586/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:15:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:15:06.949899 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6712c-6f7f-43d0-b48b-fef29a0ae4a0 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:06.960193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:07.050541 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6712c-6f7f-43d0-b48b-fef29a0ae4a0 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:07.085949 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4523f7-2825-4c86-a55f-fde2a5806802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:07.086679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4523f7-2825-4c86-a55f-fde2a5806802 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:07.086968 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4523f7-2825-4c86-a55f-fde2a5806802 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:07.087241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4523f7-2825-4c86-a55f-fde2a5806802 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:07.095300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4523f7-2825-4c86-a55f-fde2a5806802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:07.097245 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4523f7-2825-4c86-a55f-fde2a5806802 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:07.109553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-ea4523f7-2825-4c86-a55f-fde2a5806802 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 07 19:15:07.109553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:15:07.109553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 07 19:15:07.109553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 07 19:15:07.109553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:15:07.109553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:15:07.109553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:15:07.109553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:15:07.109553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 07 19:15:07.109553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:15:07.109553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:15:07.109553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:15:07.109553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:15:07.109553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:15:07.109553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:15:07.109553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:15:07.109553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:15:07.110842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:15:07.110842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:15:07.110842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:15:07.110842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:15:07.110842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:15:07.110842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:15:07.110842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:15:07.110842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:15:07.110842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:15:07.110842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:15:07.110842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:15:07.110842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:15:07.110842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:15:07.110842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:15:07.110842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:15:07.110842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:15:07.112092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:15:07.112092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:15:07.112092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:15:07.112092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:15:07.112092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:15:07.112092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:15:07.112092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:15:07.112092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:15:07.112092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:15:07.112092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:15:07.112092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 07 19:15:07.112092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:15:07.112092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 07 19:15:07.112092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:15:07.112092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 579/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Aug 7 19:15:06 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:15:07.117053 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0feb3-707e-440b-82d4-cf1c0ae4868b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:07.127843 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:07.224201 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0feb3-707e-440b-82d4-cf1c0ae4868b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:07.259946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a25d4-614b-4fdb-9000-ba0bd2ff6002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:07.260658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a25d4-614b-4fdb-9000-ba0bd2ff6002 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:07.260934 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a25d4-614b-4fdb-9000-ba0bd2ff6002 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:07.261211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a25d4-614b-4fdb-9000-ba0bd2ff6002 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:07.261852 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a25d4-614b-4fdb-9000-ba0bd2ff6002 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:15:07.262156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a25d4-614b-4fdb-9000-ba0bd2ff6002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:07.263547 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a25d4-614b-4fdb-9000-ba0bd2ff6002 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:07.271157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 587/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Sun Aug 7 19:15:07 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:09.227848 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67828982-26aa-49d4-80f4-f8b5819414ae None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:09.228473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67828982-26aa-49d4-80f4-f8b5819414ae None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:09.228473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67828982-26aa-49d4-80f4-f8b5819414ae None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:09.229755 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 580/1167] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:15:09.235298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a6408-cf42-4c45-b3b4-3417dfb97fbb None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:09.235719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a6408-cf42-4c45-b3b4-3417dfb97fbb None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:09.235836 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a6408-cf42-4c45-b3b4-3417dfb97fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:09.255689 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-b40a6408-cf42-4c45-b3b4-3417dfb97fbb None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:15:09.256923 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b40a6408-cf42-4c45-b3b4-3417dfb97fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:15:09.336958 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 588/1168] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:15:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:15:09.360172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67828982-26aa-49d4-80f4-f8b5819414ae None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:09.429364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:09.464597 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67828982-26aa-49d4-80f4-f8b5819414ae None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:09.499162 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a0db3-6735-4e2d-aa95-1a528d0a847b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:09.499785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a0db3-6735-4e2d-aa95-1a528d0a847b None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:09.500077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a0db3-6735-4e2d-aa95-1a528d0a847b None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:09.500374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a0db3-6735-4e2d-aa95-1a528d0a847b None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:09.501056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a0db3-6735-4e2d-aa95-1a528d0a847b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:09.502468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7a0db3-6735-4e2d-aa95-1a528d0a847b None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:09.522291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 581/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Sun Aug 7 19:15:09 2022] POST /identity/v3/domains => generated 267 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:15:11.528883 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f515c-e42e-4334-81f4-45a27c1584a8 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:11.529434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f515c-e42e-4334-81f4-45a27c1584a8 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:11.529434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f515c-e42e-4334-81f4-45a27c1584a8 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:11.530782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 589/1170] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:15:11.536142 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd35d18-ad81-42f5-811b-d3baf8c35833 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:11.536553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd35d18-ad81-42f5-811b-d3baf8c35833 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:11.536648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd35d18-ad81-42f5-811b-d3baf8c35833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:11.559474 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-fcd35d18-ad81-42f5-811b-d3baf8c35833 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:15:11.560740 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-fcd35d18-ad81-42f5-811b-d3baf8c35833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:15:11.629343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 582/1171] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:15:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:15:11.653032 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f515c-e42e-4334-81f4-45a27c1584a8 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:11.656153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:11.758286 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f515c-e42e-4334-81f4-45a27c1584a8 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:11.792497 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382cfdb-4c27-40d2-a84a-f9c6b9c83455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:11.793042 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382cfdb-4c27-40d2-a84a-f9c6b9c83455 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:11.793314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382cfdb-4c27-40d2-a84a-f9c6b9c83455 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:11.793584 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382cfdb-4c27-40d2-a84a-f9c6b9c83455 None admin] PATH_INFO: `/v3/domains/c777246d972f4373b78bd0fd97f41a69` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:11.804146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8382cfdb-4c27-40d2-a84a-f9c6b9c83455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c777246d972f4373b78bd0fd97f41a69)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:11.805588 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8382cfdb-4c27-40d2-a84a-f9c6b9c83455 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:11.807580 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 590/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Sun Aug 7 19:15:11 2022] GET /identity/v3/domains/c777246d972f4373b78bd0fd97f41a69 => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:11.813873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd35d18-ad81-42f5-811b-d3baf8c35833 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:11.872563 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:11.914084 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd35d18-ad81-42f5-811b-d3baf8c35833 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:11.948897 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826e653-0f91-433b-8506-d553de49cff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:11.949501 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826e653-0f91-433b-8506-d553de49cff3 None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:11.949776 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826e653-0f91-433b-8506-d553de49cff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:11.950046 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826e653-0f91-433b-8506-d553de49cff3 None admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:11.950673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826e653-0f91-433b-8506-d553de49cff3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:11.952148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826e653-0f91-433b-8506-d553de49cff3 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:11.979786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 583/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Aug 7 19:15:11 2022] POST /identity/v3/projects => generated 361 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:15:14.012652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bee2d5-87d7-46ed-b898-9b727783dd74 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:14.013156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bee2d5-87d7-46ed-b898-9b727783dd74 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:14.013156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bee2d5-87d7-46ed-b898-9b727783dd74 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:14.014565 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 591/1174] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:15:14.020173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3ba9a-ea43-467f-bcfe-fd1d364493e2 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:14.020628 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3ba9a-ea43-467f-bcfe-fd1d364493e2 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:14.020739 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3ba9a-ea43-467f-bcfe-fd1d364493e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:14.039121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-1da3ba9a-ea43-467f-bcfe-fd1d364493e2 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:15:14.040360 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1da3ba9a-ea43-467f-bcfe-fd1d364493e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:15:14.109034 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 584/1175] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:15:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:15:14.133855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bee2d5-87d7-46ed-b898-9b727783dd74 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:14.202077 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:14.242831 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bee2d5-87d7-46ed-b898-9b727783dd74 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:14.276567 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d81d4-f976-4632-9fe7-6c86fc29e6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:14.277134 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d81d4-f976-4632-9fe7-6c86fc29e6fa None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:14.277434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d81d4-f976-4632-9fe7-6c86fc29e6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:14.277707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d81d4-f976-4632-9fe7-6c86fc29e6fa None admin] PATH_INFO: `/v3/domains/c777246d972f4373b78bd0fd97f41a69` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:14.278859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d81d4-f976-4632-9fe7-6c86fc29e6fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=c777246d972f4373b78bd0fd97f41a69)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:14.280353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d81d4-f976-4632-9fe7-6c86fc29e6fa None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:14.281945 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 592/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Sun Aug 7 19:15:14 2022] GET /identity/v3/domains/c777246d972f4373b78bd0fd97f41a69 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:14.288086 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3ba9a-ea43-467f-bcfe-fd1d364493e2 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:14.332541 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:14.392219 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3ba9a-ea43-467f-bcfe-fd1d364493e2 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:14.430589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3c03d-23ec-4a04-b6c0-bae0659f2dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:14.431205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3c03d-23ec-4a04-b6c0-bae0659f2dc2 None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:14.431487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3c03d-23ec-4a04-b6c0-bae0659f2dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:14.432152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3c03d-23ec-4a04-b6c0-bae0659f2dc2 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:14.432900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b3c03d-23ec-4a04-b6c0-bae0659f2dc2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:14.434349 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b3c03d-23ec-4a04-b6c0-bae0659f2dc2 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:14.458356 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 585/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:15:14 2022] POST /identity/v3/users => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:15:16.485419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c1bdc-da77-44a1-8096-f1b8ff7fab66 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:16.486005 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c1bdc-da77-44a1-8096-f1b8ff7fab66 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:16.486005 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c1bdc-da77-44a1-8096-f1b8ff7fab66 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:16.487326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 593/1178] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:15:16.492784 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb3952-dae2-454e-b65b-974bf94bcfab None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:16.493215 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb3952-dae2-454e-b65b-974bf94bcfab None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:16.493324 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb3952-dae2-454e-b65b-974bf94bcfab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:16.512593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-0deb3952-dae2-454e-b65b-974bf94bcfab None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:15:16.513796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-0deb3952-dae2-454e-b65b-974bf94bcfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:15:16.583456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 586/1179] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:15:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:15:16.607238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c1bdc-da77-44a1-8096-f1b8ff7fab66 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:16.716379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c1bdc-da77-44a1-8096-f1b8ff7fab66 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:16.749840 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:16.756795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a36f7f-c416-4b7a-bb46-3872864ecd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:16.757433 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a36f7f-c416-4b7a-bb46-3872864ecd44 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:16.757707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a36f7f-c416-4b7a-bb46-3872864ecd44 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:16.757974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a36f7f-c416-4b7a-bb46-3872864ecd44 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:16.768628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a36f7f-c416-4b7a-bb46-3872864ecd44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:15:16.773236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-b0a36f7f-c416-4b7a-bb46-3872864ecd44 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:15:16.773236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:15:16.773236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:15:16.773236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:15:16.773236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:15:16.773236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:15:16.773236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:15:16.773236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:15:16.773236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:15:16.773236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:15:16.773236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:15:16.773236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:15:16.773236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:15:16.773236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:15:16.773236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:15:16.773236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:15:16.774482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:15:16.774482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:15:16.774482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:15:16.774482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:15:16.774482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:15:16.774482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:15:16.774482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:15:16.774482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:15:16.774482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:15:16.774482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:15:16.774482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:15:16.774482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:15:16.774482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:15:16.774482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:15:16.774482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:15:16.774482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:15:16.775406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:15:16.775406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:15:16.775406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:15:16.775406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:15:16.775406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:15:16.775406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:15:16.775406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:15:16.775406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:15:16.775406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:15:16.775406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:15:16.775406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:15:16.775406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:15:16.775406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 594/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:15:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:15:16.780875 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb3952-dae2-454e-b65b-974bf94bcfab None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:16.809324 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:16.880675 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deb3952-dae2-454e-b65b-974bf94bcfab None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:16.914870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356e88a-1e01-479d-9a79-f106b53b9cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:16.915464 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356e88a-1e01-479d-9a79-f106b53b9cef None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:16.915738 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356e88a-1e01-479d-9a79-f106b53b9cef None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:16.916034 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356e88a-1e01-479d-9a79-f106b53b9cef None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:16.916818 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6356e88a-1e01-479d-9a79-f106b53b9cef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:15:16.917136 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6356e88a-1e01-479d-9a79-f106b53b9cef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:16.918550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6356e88a-1e01-479d-9a79-f106b53b9cef None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:16.926657 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 587/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:15:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:16.933300 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a36f7f-c416-4b7a-bb46-3872864ecd44 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:17.038045 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a36f7f-c416-4b7a-bb46-3872864ecd44 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:17.073332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5f370-8c71-49ca-b0db-690b2934e507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:17.073959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5f370-8c71-49ca-b0db-690b2934e507 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:17.074236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5f370-8c71-49ca-b0db-690b2934e507 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:17.074504 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5f370-8c71-49ca-b0db-690b2934e507 None admin] PATH_INFO: `/v3/users/a97553766a6e49c6b4fcf1f31fc5e89a` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:17.088637 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5f370-8c71-49ca-b0db-690b2934e507 None admin] RBAC: Authorizing `identity:get_user(user_id=a97553766a6e49c6b4fcf1f31fc5e89a)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:17.090125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada5f370-8c71-49ca-b0db-690b2934e507 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:17.091737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 595/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:15:16 2022] GET /identity/v3/users/a97553766a6e49c6b4fcf1f31fc5e89a => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:17.097941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356e88a-1e01-479d-9a79-f106b53b9cef None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:17.190455 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:17.203624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356e88a-1e01-479d-9a79-f106b53b9cef None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:17.240022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46298c09-ff85-4fa0-a403-27fb40413cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:17.240623 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46298c09-ff85-4fa0-a403-27fb40413cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:17.240904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46298c09-ff85-4fa0-a403-27fb40413cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:17.241175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46298c09-ff85-4fa0-a403-27fb40413cf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:17.309393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46298c09-ff85-4fa0-a403-27fb40413cf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:17.310768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46298c09-ff85-4fa0-a403-27fb40413cf2 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:17.397010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:17.411100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 588/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:15:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:15:17.417336 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5f370-8c71-49ca-b0db-690b2934e507 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:17.469224 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:17.521603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5f370-8c71-49ca-b0db-690b2934e507 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:17.556931 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8a8d8-7a8b-45ac-a9ac-caf2974085d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:17.557688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8a8d8-7a8b-45ac-a9ac-caf2974085d2 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:17.557963 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8a8d8-7a8b-45ac-a9ac-caf2974085d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:17.558238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8a8d8-7a8b-45ac-a9ac-caf2974085d2 None admin] PATH_INFO: `/v3/projects/faa98632471b4e578c05b0d3e5e0af3d` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:17.559474 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8a8d8-7a8b-45ac-a9ac-caf2974085d2 None admin] RBAC: Authorizing `identity:get_project(project_id=faa98632471b4e578c05b0d3e5e0af3d)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:17.560875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8a8d8-7a8b-45ac-a9ac-caf2974085d2 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:17.562513 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 596/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:15:17 2022] GET /identity/v3/projects/faa98632471b4e578c05b0d3e5e0af3d => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:17.569043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46298c09-ff85-4fa0-a403-27fb40413cf2 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:17.669558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46298c09-ff85-4fa0-a403-27fb40413cf2 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:17.704664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9535fbb-f830-4728-983a-7a381ca908c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:17.705268 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9535fbb-f830-4728-983a-7a381ca908c6 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:17.705543 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9535fbb-f830-4728-983a-7a381ca908c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:17.705815 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9535fbb-f830-4728-983a-7a381ca908c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:17.706538 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9535fbb-f830-4728-983a-7a381ca908c6 None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=faa98632471b4e578c05b0d3e5e0af3d, user.id=a97553766a6e49c6b4fcf1f31fc5e89a) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:15:17.706861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9535fbb-f830-4728-983a-7a381ca908c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:17.708325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9535fbb-f830-4728-983a-7a381ca908c6 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:17.713971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 589/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:15:17 2022] GET /identity/v3/role_assignments?user.id=a97553766a6e49c6b4fcf1f31fc5e89a&scope.project.id=faa98632471b4e578c05b0d3e5e0af3d&role.id=4da42f08d107448fb877b42b61bc74ea => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:19.645756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260264fb-391b-47bc-a1dd-0700a28c034e None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:19.646346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260264fb-391b-47bc-a1dd-0700a28c034e None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:19.646346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260264fb-391b-47bc-a1dd-0700a28c034e None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:19.647631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 597/1186] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:15:19.653249 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50b071-bd17-4e40-9eb8-76d934878226 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:19.653685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50b071-bd17-4e40-9eb8-76d934878226 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:19.653784 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea50b071-bd17-4e40-9eb8-76d934878226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:19.672675 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-ea50b071-bd17-4e40-9eb8-76d934878226 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:15:19.673900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ea50b071-bd17-4e40-9eb8-76d934878226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:15:19.725509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:19.745951 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 590/1187] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:15:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:15:19.768529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260264fb-391b-47bc-a1dd-0700a28c034e None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:19.871792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260264fb-391b-47bc-a1dd-0700a28c034e None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:19.888744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:19.908125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3405c0-f97d-4af6-bc2e-5a5a9e6518e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:19.908698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3405c0-f97d-4af6-bc2e-5a5a9e6518e8 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:19.908974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3405c0-f97d-4af6-bc2e-5a5a9e6518e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:19.909239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3405c0-f97d-4af6-bc2e-5a5a9e6518e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:19.919685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3405c0-f97d-4af6-bc2e-5a5a9e6518e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:15:19.924318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-ac3405c0-f97d-4af6-bc2e-5a5a9e6518e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:15:19.924318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:15:19.924318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:15:19.924318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:15:19.924318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:15:19.924318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:15:19.924318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:15:19.924318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:15:19.924318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:15:19.924318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:15:19.924318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:15:19.924318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:15:19.924318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:15:19.924318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:15:19.924318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:15:19.924318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:15:19.925399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:15:19.925399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:15:19.925399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:15:19.925399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:15:19.925399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:15:19.925399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:15:19.925399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:15:19.925399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:15:19.925399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:15:19.925399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:15:19.925399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:15:19.925399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:15:19.925399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:15:19.925399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:15:19.925399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:15:19.925399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:15:19.926248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:15:19.926248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:15:19.926248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:15:19.926248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:15:19.926248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:15:19.926248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:15:19.926248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:15:19.926248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:15:19.926248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:15:19.926248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:15:19.926248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:15:19.926248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:15:19.926248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 598/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:15:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:15:19.932217 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50b071-bd17-4e40-9eb8-76d934878226 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:20.034315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea50b071-bd17-4e40-9eb8-76d934878226 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:20.070229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea892271-00d4-43ff-b19d-13fe084ad55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:20.070841 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea892271-00d4-43ff-b19d-13fe084ad55d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:20.071121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea892271-00d4-43ff-b19d-13fe084ad55d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:20.071393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea892271-00d4-43ff-b19d-13fe084ad55d None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:20.072096 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea892271-00d4-43ff-b19d-13fe084ad55d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:15:20.072400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea892271-00d4-43ff-b19d-13fe084ad55d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:20.073848 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea892271-00d4-43ff-b19d-13fe084ad55d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:20.082059 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 591/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:15:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:20.088400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3405c0-f97d-4af6-bc2e-5a5a9e6518e8 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:20.190611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3405c0-f97d-4af6-bc2e-5a5a9e6518e8 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:20.225848 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9d73d-efde-44c7-b7db-86c8b035b822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:20.226472 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9d73d-efde-44c7-b7db-86c8b035b822 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:20.226751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9d73d-efde-44c7-b7db-86c8b035b822 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:20.227021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9d73d-efde-44c7-b7db-86c8b035b822 None admin] PATH_INFO: `/v3/users/a97553766a6e49c6b4fcf1f31fc5e89a` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:20.228223 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9d73d-efde-44c7-b7db-86c8b035b822 None admin] RBAC: Authorizing `identity:get_user(user_id=a97553766a6e49c6b4fcf1f31fc5e89a)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:20.229843 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9d73d-efde-44c7-b7db-86c8b035b822 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:20.231440 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 599/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:15:20 2022] GET /identity/v3/users/a97553766a6e49c6b4fcf1f31fc5e89a => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:20.237802 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea892271-00d4-43ff-b19d-13fe084ad55d None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:20.287630 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:20.345223 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea892271-00d4-43ff-b19d-13fe084ad55d None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:20.380608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9b263-27fb-46cd-a39b-f38775697d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:20.381235 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9b263-27fb-46cd-a39b-f38775697d46 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:20.381517 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9b263-27fb-46cd-a39b-f38775697d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:20.381786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9b263-27fb-46cd-a39b-f38775697d46 None admin] PATH_INFO: `/v3/projects/faa98632471b4e578c05b0d3e5e0af3d` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:20.382978 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9b263-27fb-46cd-a39b-f38775697d46 None admin] RBAC: Authorizing `identity:get_project(project_id=faa98632471b4e578c05b0d3e5e0af3d)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:20.384446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9b263-27fb-46cd-a39b-f38775697d46 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:20.386117 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 592/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:15:20 2022] GET /identity/v3/projects/faa98632471b4e578c05b0d3e5e0af3d => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:20.392639 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9d73d-efde-44c7-b7db-86c8b035b822 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:20.440854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:20.498414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9d73d-efde-44c7-b7db-86c8b035b822 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:20.533567 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4d69c-97bd-42a7-bec5-b81b4bfbf0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:20.535260 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4d69c-97bd-42a7-bec5-b81b4bfbf0a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:20.535535 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4d69c-97bd-42a7-bec5-b81b4bfbf0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:20.535805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4d69c-97bd-42a7-bec5-b81b4bfbf0a3 None admin] PATH_INFO: `/v3/projects/faa98632471b4e578c05b0d3e5e0af3d/users/a97553766a6e49c6b4fcf1f31fc5e89a/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:20.537835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4d69c-97bd-42a7-bec5-b81b4bfbf0a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=faa98632471b4e578c05b0d3e5e0af3d, user_id=a97553766a6e49c6b4fcf1f31fc5e89a, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:20.539281 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4d69c-97bd-42a7-bec5-b81b4bfbf0a3 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:20.556919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 600/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:15:20 2022] PUT /identity/v3/projects/faa98632471b4e578c05b0d3e5e0af3d/users/a97553766a6e49c6b4fcf1f31fc5e89a/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:15:22.553619 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142de5e-a8e9-4ae6-a19d-3e52ceb8825a None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:22.554383 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142de5e-a8e9-4ae6-a19d-3e52ceb8825a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:22.554383 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142de5e-a8e9-4ae6-a19d-3e52ceb8825a None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:22.556989 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 593/1193] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:15:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:15:22.565050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81836b-722d-4c18-b7e5-d0ceaac33f6b None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:22.565795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81836b-722d-4c18-b7e5-d0ceaac33f6b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:22.565973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81836b-722d-4c18-b7e5-d0ceaac33f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:22.591340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-9e81836b-722d-4c18-b7e5-d0ceaac33f6b None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:15:22.592577 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9e81836b-722d-4c18-b7e5-d0ceaac33f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:15:22.664428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 601/1194] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:15:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:15:22.692810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142de5e-a8e9-4ae6-a19d-3e52ceb8825a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:22.803266 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142de5e-a8e9-4ae6-a19d-3e52ceb8825a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:22.817858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:22.841242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08801f9e-d7bd-4f0f-9fd8-d7e250af9785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:22.841881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08801f9e-d7bd-4f0f-9fd8-d7e250af9785 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:22.842156 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08801f9e-d7bd-4f0f-9fd8-d7e250af9785 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:22.842428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08801f9e-d7bd-4f0f-9fd8-d7e250af9785 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:22.853739 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08801f9e-d7bd-4f0f-9fd8-d7e250af9785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:15:22.858487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-08801f9e-d7bd-4f0f-9fd8-d7e250af9785 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:15:22.858487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:15:22.858487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:15:22.858487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:15:22.858487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:15:22.858487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:15:22.858487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:15:22.858487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:15:22.858487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:15:22.858487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:15:22.858487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:15:22.858487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:15:22.858487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:15:22.858487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:15:22.858487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:15:22.858487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:15:22.859683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:15:22.859683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:15:22.859683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:15:22.859683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:15:22.859683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:15:22.859683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:15:22.859683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:15:22.859683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:15:22.859683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:15:22.859683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:15:22.859683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:15:22.859683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:15:22.859683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:15:22.859683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:15:22.859683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:15:22.859683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:15:22.860558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:15:22.860558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:15:22.860558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:15:22.860558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:15:22.860558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:15:22.860558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:15:22.860558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:15:22.860558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:15:22.860558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:15:22.860558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:15:22.860558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:15:22.860558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:15:22.860558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 594/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:15:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:15:22.867797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81836b-722d-4c18-b7e5-d0ceaac33f6b None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:22.902999 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:22.977811 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81836b-722d-4c18-b7e5-d0ceaac33f6b None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:23.014909 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5959263-fced-47c5-adf4-12150eb7fe17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:23.015921 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5959263-fced-47c5-adf4-12150eb7fe17 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:23.016432 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5959263-fced-47c5-adf4-12150eb7fe17 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:23.016905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5959263-fced-47c5-adf4-12150eb7fe17 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:23.018103 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5959263-fced-47c5-adf4-12150eb7fe17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:15:23.018560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5959263-fced-47c5-adf4-12150eb7fe17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:23.020020 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5959263-fced-47c5-adf4-12150eb7fe17 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:23.030309 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 602/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:15:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:23.036589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08801f9e-d7bd-4f0f-9fd8-d7e250af9785 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:23.149104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08801f9e-d7bd-4f0f-9fd8-d7e250af9785 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:23.186829 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603f7fb-e348-469c-8e72-4203ac2d64d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:23.187457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603f7fb-e348-469c-8e72-4203ac2d64d5 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:23.187736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603f7fb-e348-469c-8e72-4203ac2d64d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:23.188040 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603f7fb-e348-469c-8e72-4203ac2d64d5 None admin] PATH_INFO: `/v3/users/a97553766a6e49c6b4fcf1f31fc5e89a` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:23.189259 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603f7fb-e348-469c-8e72-4203ac2d64d5 None admin] RBAC: Authorizing `identity:get_user(user_id=a97553766a6e49c6b4fcf1f31fc5e89a)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:23.190691 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603f7fb-e348-469c-8e72-4203ac2d64d5 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:23.192319 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 595/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Sun Aug 7 19:15:23 2022] GET /identity/v3/users/a97553766a6e49c6b4fcf1f31fc5e89a => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:23.198542 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5959263-fced-47c5-adf4-12150eb7fe17 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:23.302700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:23.309896 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5959263-fced-47c5-adf4-12150eb7fe17 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:23.349100 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a70c6-c486-40fb-8cfc-0d66881aaf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:23.349708 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a70c6-c486-40fb-8cfc-0d66881aaf97 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:23.349984 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a70c6-c486-40fb-8cfc-0d66881aaf97 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:23.350257 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a70c6-c486-40fb-8cfc-0d66881aaf97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:23.424426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a70c6-c486-40fb-8cfc-0d66881aaf97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:23.425893 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a70c6-c486-40fb-8cfc-0d66881aaf97 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:23.523088 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:23.533285 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 603/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:15:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:15:23.539411 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603f7fb-e348-469c-8e72-4203ac2d64d5 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:23.587058 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:23.657416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603f7fb-e348-469c-8e72-4203ac2d64d5 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:23.693501 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0bb28-c814-46e7-a2a1-b4e87900dbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:23.694082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0bb28-c814-46e7-a2a1-b4e87900dbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:23.694361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0bb28-c814-46e7-a2a1-b4e87900dbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:23.694639 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0bb28-c814-46e7-a2a1-b4e87900dbe0 None admin] PATH_INFO: `/v3/projects/faa98632471b4e578c05b0d3e5e0af3d` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:23.695837 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0bb28-c814-46e7-a2a1-b4e87900dbe0 None admin] RBAC: Authorizing `identity:get_project(project_id=faa98632471b4e578c05b0d3e5e0af3d)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:23.697313 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0bb28-c814-46e7-a2a1-b4e87900dbe0 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:23.699048 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 596/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Sun Aug 7 19:15:23 2022] GET /identity/v3/projects/faa98632471b4e578c05b0d3e5e0af3d => generated 361 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:23.705415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a70c6-c486-40fb-8cfc-0d66881aaf97 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:15:23.818016 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a70c6-c486-40fb-8cfc-0d66881aaf97 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:15:23.855912 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384b1fa-8d62-4628-8729-6ff3be20789d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:15:23.856570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384b1fa-8d62-4628-8729-6ff3be20789d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:15:23.856848 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384b1fa-8d62-4628-8729-6ff3be20789d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:15:23.857131 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384b1fa-8d62-4628-8729-6ff3be20789d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:15:23.857862 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384b1fa-8d62-4628-8729-6ff3be20789d None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=faa98632471b4e578c05b0d3e5e0af3d, user.id=a97553766a6e49c6b4fcf1f31fc5e89a) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:15:23.858171 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384b1fa-8d62-4628-8729-6ff3be20789d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:15:23.859611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9384b1fa-8d62-4628-8729-6ff3be20789d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:15:23.867483 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 604/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:15:23 2022] GET /identity/v3/role_assignments?user.id=a97553766a6e49c6b4fcf1f31fc5e89a&scope.project.id=faa98632471b4e578c05b0d3e5e0af3d&role.id=4da42f08d107448fb877b42b61bc74ea => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:15:33.688462 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=46 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:15:33.864529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=70 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:19.396421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a1c56-2906-4621-8898-b749ed3f1b1e None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:19.396996 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a1c56-2906-4621-8898-b749ed3f1b1e None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:19.396996 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a1c56-2906-4621-8898-b749ed3f1b1e None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:19.398317 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 597/1201] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:16:19.405832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0b7a9-11c8-4910-ae28-747c7265d2bc None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:19.406441 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0b7a9-11c8-4910-ae28-747c7265d2bc None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:19.406608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0b7a9-11c8-4910-ae28-747c7265d2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:19.433334 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-8fd0b7a9-11c8-4910-ae28-747c7265d2bc None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:16:19.434949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8fd0b7a9-11c8-4910-ae28-747c7265d2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:16:19.528686 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 605/1202] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:16:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 19:16:19.557610 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a1c56-2906-4621-8898-b749ed3f1b1e None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:19.738342 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a1c56-2906-4621-8898-b749ed3f1b1e None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:19.784181 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ff0c4-1e13-4776-a523-1e8b2f861337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:19.784846 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ff0c4-1e13-4776-a523-1e8b2f861337 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:19.785154 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ff0c4-1e13-4776-a523-1e8b2f861337 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:19.785392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ff0c4-1e13-4776-a523-1e8b2f861337 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:19.796944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ff0c4-1e13-4776-a523-1e8b2f861337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:19.798409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ff0c4-1e13-4776-a523-1e8b2f861337 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:19.815677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-5c3ff0c4-1e13-4776-a523-1e8b2f861337 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:19.815677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:19.815677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:19.815677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:19.815677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:19.815677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:19.815677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:19.815677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:19.815677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:19.815677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:19.815677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:19.815677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:19.815677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:16:19.815677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:16:19.815677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:16:19.815677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:16:19.820115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:19.820115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:19.820115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:19.820115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:19.820115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:19.820115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:19.820115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:19.820115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:19.820115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:19.820115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:19.820115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:19.820115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:19.820115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:19.820115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:19.820115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:19.820115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:19.820115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:16:19.821368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:16:19.821368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:19.821368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:16:19.821368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 598/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:19.823460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0b7a9-11c8-4910-ae28-747c7265d2bc None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:19.943164 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0b7a9-11c8-4910-ae28-747c7265d2bc None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:19.982584 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:19.990961 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af8519-d397-4956-a3b7-c2b5672adc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:19.991623 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af8519-d397-4956-a3b7-c2b5672adc62 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:19.992123 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af8519-d397-4956-a3b7-c2b5672adc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:19.992522 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af8519-d397-4956-a3b7-c2b5672adc62 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:19.993615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af8519-d397-4956-a3b7-c2b5672adc62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:19.994126 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af8519-d397-4956-a3b7-c2b5672adc62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:19.996735 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af8519-d397-4956-a3b7-c2b5672adc62 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:20.016031 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 606/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:20.022116 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ff0c4-1e13-4776-a523-1e8b2f861337 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:20.093225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:20.166698 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ff0c4-1e13-4776-a523-1e8b2f861337 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:20.212067 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddb069-dbcd-4bf5-8a0d-1a82a1620003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:20.212791 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddb069-dbcd-4bf5-8a0d-1a82a1620003 None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:20.213072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddb069-dbcd-4bf5-8a0d-1a82a1620003 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:20.213440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddb069-dbcd-4bf5-8a0d-1a82a1620003 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:20.214225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ddb069-dbcd-4bf5-8a0d-1a82a1620003 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:20.215964 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ddb069-dbcd-4bf5-8a0d-1a82a1620003 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:20.245635 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 599/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Sun Aug 7 19:16:20 2022] POST /identity/v3/users => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:16:22.746263 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a355ec6-920e-47e5-aa8f-f147d99a1ee1 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:22.746867 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a355ec6-920e-47e5-aa8f-f147d99a1ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:22.746867 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a355ec6-920e-47e5-aa8f-f147d99a1ee1 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:22.748472 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 607/1206] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:16:22.754272 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0dd79-73ac-4d8b-9b1b-2461bc9cbd3e None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:22.754777 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0dd79-73ac-4d8b-9b1b-2461bc9cbd3e None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:22.754950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0dd79-73ac-4d8b-9b1b-2461bc9cbd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:22.777088 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-8fd0dd79-73ac-4d8b-9b1b-2461bc9cbd3e None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:16:22.778654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8fd0dd79-73ac-4d8b-9b1b-2461bc9cbd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:16:22.884179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 600/1207] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:16:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:16:22.910419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a355ec6-920e-47e5-aa8f-f147d99a1ee1 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:23.062190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a355ec6-920e-47e5-aa8f-f147d99a1ee1 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:23.123437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b964c1-5ca3-41b3-9d24-442853807f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:23.124583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b964c1-5ca3-41b3-9d24-442853807f07 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:23.125100 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b964c1-5ca3-41b3-9d24-442853807f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:23.125577 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b964c1-5ca3-41b3-9d24-442853807f07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:23.140816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b964c1-5ca3-41b3-9d24-442853807f07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:16:23.145548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-09b964c1-5ca3-41b3-9d24-442853807f07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:16:23.145548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:23.145548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:23.145548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:23.145548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:23.145548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:23.145548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:23.145548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:23.145548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:23.145548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:23.145548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:23.145548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:23.145548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:16:23.145548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:16:23.145548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:16:23.145548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:16:23.146579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:16:23.146579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:16:23.146579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:16:23.146579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:16:23.146579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:23.146579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:23.146579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:23.146579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:23.146579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:23.146579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:23.146579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:23.146579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:23.146579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:23.146579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:23.146579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:23.146579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:23.148490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:23.148490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:23.148490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:23.148490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:23.148490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:16:23.148490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:16:23.148490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:16:23.148490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:16:23.148490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:16:23.148490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:16:23.148490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:16:23.148490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:23.148490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 608/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:16:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:23.157611 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0dd79-73ac-4d8b-9b1b-2461bc9cbd3e None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:23.216677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:23.306002 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0dd79-73ac-4d8b-9b1b-2461bc9cbd3e None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:23.345607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c731072-ff2e-4215-af72-606c260038a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:23.346487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c731072-ff2e-4215-af72-606c260038a4 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:23.346959 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c731072-ff2e-4215-af72-606c260038a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:23.347355 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c731072-ff2e-4215-af72-606c260038a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:23.348236 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c731072-ff2e-4215-af72-606c260038a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:23.348539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c731072-ff2e-4215-af72-606c260038a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:23.349901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c731072-ff2e-4215-af72-606c260038a4 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:23.360034 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 601/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:16:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:16:23.365500 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b964c1-5ca3-41b3-9d24-442853807f07 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:23.421361 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:23.485572 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b964c1-5ca3-41b3-9d24-442853807f07 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:23.528920 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1da7f-f41b-48e0-9202-1ae11a86389b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:23.529649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1da7f-f41b-48e0-9202-1ae11a86389b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:23.529956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1da7f-f41b-48e0-9202-1ae11a86389b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:23.530238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1da7f-f41b-48e0-9202-1ae11a86389b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:23.544302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1da7f-f41b-48e0-9202-1ae11a86389b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:23.545841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1da7f-f41b-48e0-9202-1ae11a86389b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:23.565581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-e8e1da7f-f41b-48e0-9202-1ae11a86389b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:23.565581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:23.565581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:23.565581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:23.565581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:23.565581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:23.565581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:23.565581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:23.565581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:23.565581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:23.565581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:23.565581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:23.565581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:16:23.565581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:16:23.565581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:16:23.565581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:16:23.566834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:23.566834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:23.566834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:23.566834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:23.566834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:23.566834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:23.566834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:23.566834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:23.566834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:23.566834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:23.566834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:23.566834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:23.566834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:23.566834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:23.566834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:23.566834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:23.566834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:16:23.567719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:16:23.567719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:23.567719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:23.567719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 609/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:16:23.573096 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c731072-ff2e-4215-af72-606c260038a4 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:23.679878 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:23.683464 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c731072-ff2e-4215-af72-606c260038a4 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:23.721650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4f898-899c-474b-abeb-933e87cd5860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:23.722418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4f898-899c-474b-abeb-933e87cd5860 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:23.722841 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4f898-899c-474b-abeb-933e87cd5860 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:23.723062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4f898-899c-474b-abeb-933e87cd5860 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:23.723696 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4f898-899c-474b-abeb-933e87cd5860 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:23.724170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4f898-899c-474b-abeb-933e87cd5860 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:23.725888 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4f898-899c-474b-abeb-933e87cd5860 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:23.740109 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 602/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:23.745940 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1da7f-f41b-48e0-9202-1ae11a86389b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:23.856673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1da7f-f41b-48e0-9202-1ae11a86389b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:23.859388 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:23.900811 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec21d4-0c54-422a-9119-673f7841c2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:23.901651 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec21d4-0c54-422a-9119-673f7841c2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:23.901928 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec21d4-0c54-422a-9119-673f7841c2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:23.902201 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec21d4-0c54-422a-9119-673f7841c2e9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:23.911504 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec21d4-0c54-422a-9119-673f7841c2e9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:23.913246 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec21d4-0c54-422a-9119-673f7841c2e9 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:23.929593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-80ec21d4-0c54-422a-9119-673f7841c2e9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 19:16:23.929593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:23.929593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:23.929593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:23.929593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:23.929593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:23.929593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:23.929593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:23.929593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:23.929593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:23.929593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:23.929593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:23.929593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:16:23.929593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:16:23.929593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:16:23.929593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:23.936765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:23.938170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:23.938170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:23.938170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:16:23.938170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:16:23.938170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:16:23.938170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:16:23.938170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:16:23.938170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:16:23.938170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 19:16:23.938170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:23.938170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 610/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:23.942732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4f898-899c-474b-abeb-933e87cd5860 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:24.070331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4f898-899c-474b-abeb-933e87cd5860 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:24.108053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab5a7d-48e4-4352-960f-91d4de2ff792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:24.108656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab5a7d-48e4-4352-960f-91d4de2ff792 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:24.108976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab5a7d-48e4-4352-960f-91d4de2ff792 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:24.109263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab5a7d-48e4-4352-960f-91d4de2ff792 None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:24.110028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab5a7d-48e4-4352-960f-91d4de2ff792 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:24.110348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab5a7d-48e4-4352-960f-91d4de2ff792 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:24.111926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab5a7d-48e4-4352-960f-91d4de2ff792 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:24.137851 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 603/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Sun Aug 7 19:16:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:24.139263 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec21d4-0c54-422a-9119-673f7841c2e9 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:24.260938 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec21d4-0c54-422a-9119-673f7841c2e9 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:24.298043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73b04b-3d3e-4490-88b2-9b92d172504f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:24.298704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73b04b-3d3e-4490-88b2-9b92d172504f None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:24.298983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73b04b-3d3e-4490-88b2-9b92d172504f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:24.299249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73b04b-3d3e-4490-88b2-9b92d172504f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:24.334333 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:24.387762 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73b04b-3d3e-4490-88b2-9b92d172504f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:24.389248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d73b04b-3d3e-4490-88b2-9b92d172504f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:24.510722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 611/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:16:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:16:24.516937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab5a7d-48e4-4352-960f-91d4de2ff792 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:24.536909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:24.625463 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab5a7d-48e4-4352-960f-91d4de2ff792 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:24.661938 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a094da4-763d-401d-9599-abf97506111d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:24.662591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a094da4-763d-401d-9599-abf97506111d None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:24.662868 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a094da4-763d-401d-9599-abf97506111d None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:24.663148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a094da4-763d-401d-9599-abf97506111d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:24.675205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a094da4-763d-401d-9599-abf97506111d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:24.676836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a094da4-763d-401d-9599-abf97506111d None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:24.693731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-3a094da4-763d-401d-9599-abf97506111d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:24.693731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:24.693731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:24.693731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:24.693731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:24.693731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:24.693731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:24.693731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:24.693731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:24.693731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:24.693731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:24.693731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:24.693731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:16:24.693731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:16:24.693731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:16:24.693731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:16:24.694800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:24.694800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:24.694800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:24.694800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:24.694800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:24.694800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:24.694800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:24.694800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:24.694800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:24.694800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:24.694800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:24.694800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:24.694800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:24.694800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:24.694800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:24.694800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:24.694800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:16:24.695702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:16:24.695702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:24.695702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:16:24.700028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 604/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:24.701171 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73b04b-3d3e-4490-88b2-9b92d172504f None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:24.777682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:24.822269 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73b04b-3d3e-4490-88b2-9b92d172504f None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:24.863682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26016fa7-c146-4a0e-b23f-7f34d0509850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:24.864439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26016fa7-c146-4a0e-b23f-7f34d0509850 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:24.864790 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26016fa7-c146-4a0e-b23f-7f34d0509850 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:24.865121 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26016fa7-c146-4a0e-b23f-7f34d0509850 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:24.865867 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26016fa7-c146-4a0e-b23f-7f34d0509850 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:24.866242 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26016fa7-c146-4a0e-b23f-7f34d0509850 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:24.867733 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26016fa7-c146-4a0e-b23f-7f34d0509850 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:24.880414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 612/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:24.886314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a094da4-763d-401d-9599-abf97506111d None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:24.965089 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:24.998950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a094da4-763d-401d-9599-abf97506111d None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:25.036593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f4c70-9f8f-45be-84a8-c2a7d22bd7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:25.037401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f4c70-9f8f-45be-84a8-c2a7d22bd7a8 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:25.037749 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f4c70-9f8f-45be-84a8-c2a7d22bd7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:25.038122 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f4c70-9f8f-45be-84a8-c2a7d22bd7a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:25.047633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f4c70-9f8f-45be-84a8-c2a7d22bd7a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:25.049230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f4c70-9f8f-45be-84a8-c2a7d22bd7a8 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:25.061300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-840f4c70-9f8f-45be-84a8-c2a7d22bd7a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:16:25.061300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:25.061300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:25.061300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:25.061300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:25.061300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:25.061300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:25.061300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:25.061300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:25.061300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:25.061300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:25.061300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:25.061300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:16:25.061300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:16:25.061300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:16:25.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:16:25.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:25.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:25.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:25.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:25.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:25.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:25.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:25.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:25.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:25.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:25.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:25.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:25.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:25.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:25.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:25.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:25.063648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:16:25.063648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:16:25.063648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:16:25.063648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:16:25.063648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:16:25.063648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:16:25.063648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:16:25.063648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:16:25.070894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26016fa7-c146-4a0e-b23f-7f34d0509850 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:25.071438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 605/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:16:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:25.187828 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26016fa7-c146-4a0e-b23f-7f34d0509850 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:25.214962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:25.232291 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc5ce0-5a96-4dc6-9a26-4600c457d07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:25.232837 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc5ce0-5a96-4dc6-9a26-4600c457d07b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:25.233111 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc5ce0-5a96-4dc6-9a26-4600c457d07b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:25.233384 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc5ce0-5a96-4dc6-9a26-4600c457d07b None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:25.234088 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc5ce0-5a96-4dc6-9a26-4600c457d07b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:25.234396 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc5ce0-5a96-4dc6-9a26-4600c457d07b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:25.235854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc5ce0-5a96-4dc6-9a26-4600c457d07b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:25.250441 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 613/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:16:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:25.257396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f4c70-9f8f-45be-84a8-c2a7d22bd7a8 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:25.363105 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f4c70-9f8f-45be-84a8-c2a7d22bd7a8 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:25.396926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:25.405300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0518c-6830-4567-8fff-fb4ef2508195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:25.405921 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0518c-6830-4567-8fff-fb4ef2508195 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:25.406204 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0518c-6830-4567-8fff-fb4ef2508195 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:25.406481 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0518c-6830-4567-8fff-fb4ef2508195 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:25.407221 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0518c-6830-4567-8fff-fb4ef2508195 None admin] RBAC: Adding query filter params (role.id=43aa4a9830054a09baa1f0aecfd125e2, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=143c30b4201645279564e72ab3288adc) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:25.407528 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0518c-6830-4567-8fff-fb4ef2508195 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:25.409244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad0518c-6830-4567-8fff-fb4ef2508195 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:25.424421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 606/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:16:25 2022] GET /identity/v3/role_assignments?user.id=143c30b4201645279564e72ab3288adc&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=43aa4a9830054a09baa1f0aecfd125e2 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:27.561141 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaed809-34e8-4bb5-b6b3-079fafc1172a None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:27.561141 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaed809-34e8-4bb5-b6b3-079fafc1172a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:27.561141 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaed809-34e8-4bb5-b6b3-079fafc1172a None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:27.561141 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 614/1220] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:16:27.566692 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03288b1-0127-4943-a4e7-4c124cb86b07 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:27.567160 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03288b1-0127-4943-a4e7-4c124cb86b07 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:27.567269 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03288b1-0127-4943-a4e7-4c124cb86b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:27.599304 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-e03288b1-0127-4943-a4e7-4c124cb86b07 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:16:27.600668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e03288b1-0127-4943-a4e7-4c124cb86b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:16:27.674094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 607/1221] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:16:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:16:27.705915 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaed809-34e8-4bb5-b6b3-079fafc1172a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:27.827274 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaed809-34e8-4bb5-b6b3-079fafc1172a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:27.862131 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9c8a6-2c20-449d-b52c-76f0670704d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:27.862801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9c8a6-2c20-449d-b52c-76f0670704d3 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:27.863080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9c8a6-2c20-449d-b52c-76f0670704d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:27.863363 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9c8a6-2c20-449d-b52c-76f0670704d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:27.874285 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde9c8a6-2c20-449d-b52c-76f0670704d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:16:27.879230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-dde9c8a6-2c20-449d-b52c-76f0670704d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:16:27.879230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:27.879230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:27.879230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:27.879230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:27.879230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:27.879230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:27.879230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:27.879230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:27.879230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:27.879230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:27.879230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:27.879230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:16:27.879230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:16:27.879230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:16:27.879230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:16:27.889679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:16:27.889679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:16:27.889679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:16:27.889679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:16:27.889679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:27.889679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:27.889679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:27.889679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:27.889679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:27.889679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:27.889679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:27.889679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:27.889679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:27.889679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:27.889679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:27.889679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:27.890802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:27.890802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:27.890802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:27.890802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:27.890802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:16:27.890802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:16:27.890802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:16:27.890802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:16:27.890802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:16:27.890802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:16:27.890802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:16:27.890802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:27.890802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 615/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:16:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:27.892279 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03288b1-0127-4943-a4e7-4c124cb86b07 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:28.025135 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03288b1-0127-4943-a4e7-4c124cb86b07 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:28.050795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:28.065780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fb3e0-ac70-4e9e-a796-30550fe9ee60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:28.066402 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fb3e0-ac70-4e9e-a796-30550fe9ee60 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:28.066681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fb3e0-ac70-4e9e-a796-30550fe9ee60 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:28.066950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fb3e0-ac70-4e9e-a796-30550fe9ee60 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:28.067626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fb3e0-ac70-4e9e-a796-30550fe9ee60 None admin] RBAC: Adding query filter params (name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:28.067929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fb3e0-ac70-4e9e-a796-30550fe9ee60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:28.069573 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fb3e0-ac70-4e9e-a796-30550fe9ee60 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:28.081897 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 608/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:16:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:28.089371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9c8a6-2c20-449d-b52c-76f0670704d3 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:28.140872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:28.207473 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9c8a6-2c20-449d-b52c-76f0670704d3 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:28.243562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee429b1-6592-4f7d-9814-4e47394fa24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:28.244209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee429b1-6592-4f7d-9814-4e47394fa24f None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:28.244513 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee429b1-6592-4f7d-9814-4e47394fa24f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:28.244786 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee429b1-6592-4f7d-9814-4e47394fa24f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:28.255123 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee429b1-6592-4f7d-9814-4e47394fa24f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:28.256631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee429b1-6592-4f7d-9814-4e47394fa24f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:28.271914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-0ee429b1-6592-4f7d-9814-4e47394fa24f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:28.271914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:28.271914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:28.271914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:28.271914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:28.271914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:28.271914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:28.271914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:28.271914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:28.271914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:28.271914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:28.271914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:28.271914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:16:28.271914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:16:28.271914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:16:28.271914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:16:28.273055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:28.273055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:28.273055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:28.273055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:28.273055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:28.273055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:28.273055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:28.273055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:28.273055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:28.273055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:28.273055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:28.273055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:28.273055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:28.273055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:28.273055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:28.273055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:28.273055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:16:28.274439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:16:28.274439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:28.274439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:28.274439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 616/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:28.279797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fb3e0-ac70-4e9e-a796-30550fe9ee60 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:28.392961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fb3e0-ac70-4e9e-a796-30550fe9ee60 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:28.427969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a9da8-8c3d-4a66-8126-60aa0cb13a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:28.428616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a9da8-8c3d-4a66-8126-60aa0cb13a90 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:28.428896 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a9da8-8c3d-4a66-8126-60aa0cb13a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:28.429166 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a9da8-8c3d-4a66-8126-60aa0cb13a90 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:28.429809 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a9da8-8c3d-4a66-8126-60aa0cb13a90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:28.430111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a9da8-8c3d-4a66-8126-60aa0cb13a90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:28.431838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a9da8-8c3d-4a66-8126-60aa0cb13a90 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:28.444782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 609/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:28.450801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee429b1-6592-4f7d-9814-4e47394fa24f None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:28.558325 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:28.566552 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee429b1-6592-4f7d-9814-4e47394fa24f None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:28.604831 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c7408-80bb-4637-82cd-d481cdbe1e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:28.605579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c7408-80bb-4637-82cd-d481cdbe1e65 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:28.605878 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c7408-80bb-4637-82cd-d481cdbe1e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:28.606148 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c7408-80bb-4637-82cd-d481cdbe1e65 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:28.614323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c7408-80bb-4637-82cd-d481cdbe1e65 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:28.615876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1c7408-80bb-4637-82cd-d481cdbe1e65 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:28.627236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-8a1c7408-80bb-4637-82cd-d481cdbe1e65 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 19:16:28.627236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:28.627236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:28.627236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:28.627236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:28.627236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:28.627236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:28.627236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:28.627236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:28.627236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:28.627236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:28.627236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:28.627236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:16:28.627236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:16:28.627236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:16:28.627236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:28.628459 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:28.629725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:28.629725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:28.629725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:16:28.629725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:16:28.629725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:16:28.629725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:16:28.629725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:16:28.629725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:16:28.629725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 19:16:28.629725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:28.632318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 617/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:28.634299 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a9da8-8c3d-4a66-8126-60aa0cb13a90 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:28.689735 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:28.764561 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a9da8-8c3d-4a66-8126-60aa0cb13a90 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:28.805075 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fd5ed-3645-44c5-9d9f-e13cb794f96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:28.805710 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fd5ed-3645-44c5-9d9f-e13cb794f96e None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:28.805990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fd5ed-3645-44c5-9d9f-e13cb794f96e None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:28.806686 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fd5ed-3645-44c5-9d9f-e13cb794f96e None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:28.807480 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fd5ed-3645-44c5-9d9f-e13cb794f96e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:28.807792 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fd5ed-3645-44c5-9d9f-e13cb794f96e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:28.809333 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fd5ed-3645-44c5-9d9f-e13cb794f96e None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:28.827886 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 610/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Sun Aug 7 19:16:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:28.833988 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c7408-80bb-4637-82cd-d481cdbe1e65 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:28.949331 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1c7408-80bb-4637-82cd-d481cdbe1e65 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:28.985656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e83c7b-b71b-41d0-ac26-d3d7ad60734d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:28.986291 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e83c7b-b71b-41d0-ac26-d3d7ad60734d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:28.986565 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e83c7b-b71b-41d0-ac26-d3d7ad60734d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:28.986831 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e83c7b-b71b-41d0-ac26-d3d7ad60734d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:28.997668 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e83c7b-b71b-41d0-ac26-d3d7ad60734d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:28.999248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e83c7b-b71b-41d0-ac26-d3d7ad60734d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:29.015049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-80e83c7b-b71b-41d0-ac26-d3d7ad60734d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:29.015049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:29.015049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:29.015049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:29.015049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:29.015049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:29.015049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:29.015049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:29.015049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:29.015049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:29.015049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:29.015049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:29.015049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:16:29.015049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:16:29.015049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:16:29.015049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:16:29.016526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:29.016526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:29.016526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:29.016526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:29.016526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:29.016526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:29.016526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:29.016526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:29.016526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:29.016526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:29.016526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:29.016526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:29.016526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:29.016526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:29.016526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:29.016526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:29.016526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:16:29.017454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:16:29.017454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:29.017454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:29.017454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 618/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:29.022350 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fd5ed-3645-44c5-9d9f-e13cb794f96e None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:29.133033 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fd5ed-3645-44c5-9d9f-e13cb794f96e None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:29.135741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:29.171299 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e92271-c5ec-40fe-9249-d6900fe2d32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:29.171923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e92271-c5ec-40fe-9249-d6900fe2d32e None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:29.172243 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e92271-c5ec-40fe-9249-d6900fe2d32e None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:29.172508 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e92271-c5ec-40fe-9249-d6900fe2d32e None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:29.173159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e92271-c5ec-40fe-9249-d6900fe2d32e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:29.173510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e92271-c5ec-40fe-9249-d6900fe2d32e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:29.174926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e92271-c5ec-40fe-9249-d6900fe2d32e None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:29.188108 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 611/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:29.194213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e83c7b-b71b-41d0-ac26-d3d7ad60734d None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:29.205744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:29.317615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e83c7b-b71b-41d0-ac26-d3d7ad60734d None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:29.357872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254d165-4f97-4573-a154-8ec8b0abb1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:29.358511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254d165-4f97-4573-a154-8ec8b0abb1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:29.358785 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254d165-4f97-4573-a154-8ec8b0abb1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:29.359050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254d165-4f97-4573-a154-8ec8b0abb1e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:29.366018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254d165-4f97-4573-a154-8ec8b0abb1e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:29.367472 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7254d165-4f97-4573-a154-8ec8b0abb1e4 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:29.379180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-7254d165-4f97-4573-a154-8ec8b0abb1e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:16:29.379180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:29.379180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:29.379180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:29.379180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:29.379180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:29.379180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:29.379180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:29.379180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:29.379180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:29.379180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:29.379180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:29.379180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:16:29.379180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:16:29.379180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:16:29.380243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:16:29.380243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:29.380243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:29.380243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:29.380243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:29.380243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:29.380243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:29.380243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:29.380243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:29.380243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:29.380243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:29.380243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:29.380243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:29.380243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:29.380243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:29.380243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:29.380243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:29.381145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:16:29.381145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:16:29.381145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:16:29.381145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:16:29.381145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:16:29.381145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:16:29.381145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:16:29.381145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:29.381145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 619/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:16:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:29.386488 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e92271-c5ec-40fe-9249-d6900fe2d32e None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:29.489070 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e92271-c5ec-40fe-9249-d6900fe2d32e None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:29.524274 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5cfe7-d5ce-4edc-b44a-789927890721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:29.524892 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5cfe7-d5ce-4edc-b44a-789927890721 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:29.525167 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5cfe7-d5ce-4edc-b44a-789927890721 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:29.525434 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5cfe7-d5ce-4edc-b44a-789927890721 None admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:29.526224 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5cfe7-d5ce-4edc-b44a-789927890721 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:29.526532 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5cfe7-d5ce-4edc-b44a-789927890721 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:29.528039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d5cfe7-d5ce-4edc-b44a-789927890721 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:29.540353 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 612/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:16:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:29.546540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254d165-4f97-4573-a154-8ec8b0abb1e4 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:29.618549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:29.664766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254d165-4f97-4573-a154-8ec8b0abb1e4 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:29.702738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1381933-bb6e-4eb5-9132-2121335caf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:29.703418 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1381933-bb6e-4eb5-9132-2121335caf07 None admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:29.703704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1381933-bb6e-4eb5-9132-2121335caf07 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:29.703973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1381933-bb6e-4eb5-9132-2121335caf07 None admin] PATH_INFO: `/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/143c30b4201645279564e72ab3288adc/roles/43aa4a9830054a09baa1f0aecfd125e2` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:29.722113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1381933-bb6e-4eb5-9132-2121335caf07 None admin] RBAC: Authorizing `identity:create_grant(project_id=37703d50cd7e4d9bbdc797a7a1466b66, user_id=143c30b4201645279564e72ab3288adc, role_id=43aa4a9830054a09baa1f0aecfd125e2)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:29.724005 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1381933-bb6e-4eb5-9132-2121335caf07 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:29.740285 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 620/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:16:29 2022] PUT /identity/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/143c30b4201645279564e72ab3288adc/roles/43aa4a9830054a09baa1f0aecfd125e2 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:16:31.919301 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2757b-728a-4393-a3af-20f60c69a2dd None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:31.919871 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2757b-728a-4393-a3af-20f60c69a2dd None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:31.919960 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2757b-728a-4393-a3af-20f60c69a2dd None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:31.921265 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 613/1233] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:16:31.926778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecf89b-b2e8-4749-ad21-a30800491c2c None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:31.927192 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecf89b-b2e8-4749-ad21-a30800491c2c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:31.927360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecf89b-b2e8-4749-ad21-a30800491c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:31.948856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-a3ecf89b-b2e8-4749-ad21-a30800491c2c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:16:31.950251 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a3ecf89b-b2e8-4749-ad21-a30800491c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:16:32.024871 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 621/1234] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:16:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:16:32.049497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2757b-728a-4393-a3af-20f60c69a2dd None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:32.081511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:32.171929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2757b-728a-4393-a3af-20f60c69a2dd None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:32.210880 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27208119-1525-4065-8fb4-1f23649f5edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:32.211580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27208119-1525-4065-8fb4-1f23649f5edb None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:32.211887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27208119-1525-4065-8fb4-1f23649f5edb None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:32.212200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27208119-1525-4065-8fb4-1f23649f5edb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:32.226012 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27208119-1525-4065-8fb4-1f23649f5edb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:16:32.231936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-27208119-1525-4065-8fb4-1f23649f5edb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:16:32.231936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:32.231936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:32.231936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:32.231936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:32.231936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:32.231936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:32.231936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:32.231936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:32.231936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:32.231936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:32.231936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:32.231936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:16:32.231936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:16:32.231936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:16:32.231936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:16:32.233073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:16:32.233073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:16:32.233073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:16:32.233073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:16:32.233073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:32.233073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:32.233073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:32.233073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:32.233073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:32.233073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:32.233073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:32.233073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:32.233073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:32.233073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:32.233073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:32.233073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:32.239191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:32.239191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:32.239191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:32.239191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:32.239191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:16:32.239191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:16:32.239191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:16:32.239191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:16:32.239191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:16:32.239191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:16:32.239191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 19:16:32.239191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:16:32.239191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 614/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Sun Aug 7 19:16:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:32.241079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecf89b-b2e8-4749-ad21-a30800491c2c None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:32.291964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:32.355211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecf89b-b2e8-4749-ad21-a30800491c2c None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:32.395612 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5418bbc-286b-4f33-addb-e822738ecb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:32.396264 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5418bbc-286b-4f33-addb-e822738ecb43 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:32.396591 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5418bbc-286b-4f33-addb-e822738ecb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:32.396894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5418bbc-286b-4f33-addb-e822738ecb43 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:32.397788 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5418bbc-286b-4f33-addb-e822738ecb43 None admin] RBAC: Adding query filter params (name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:32.398241 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5418bbc-286b-4f33-addb-e822738ecb43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:32.399780 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5418bbc-286b-4f33-addb-e822738ecb43 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:32.414752 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 622/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Sun Aug 7 19:16:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:32.416747 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27208119-1525-4065-8fb4-1f23649f5edb None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:32.517985 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:32.537228 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27208119-1525-4065-8fb4-1f23649f5edb None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:32.579590 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8576b-d123-4152-8720-d6c1c3eef1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:32.580275 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8576b-d123-4152-8720-d6c1c3eef1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:32.580549 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8576b-d123-4152-8720-d6c1c3eef1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:32.580814 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8576b-d123-4152-8720-d6c1c3eef1a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:32.591705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8576b-d123-4152-8720-d6c1c3eef1a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:32.593421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8576b-d123-4152-8720-d6c1c3eef1a9 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:32.612546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-18b8576b-d123-4152-8720-d6c1c3eef1a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:32.612546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:32.612546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:32.612546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:32.612546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:32.612546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:32.612546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:32.612546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:32.612546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:32.612546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:32.612546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:32.612546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:32.612546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:16:32.612546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:16:32.612546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:16:32.612546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:16:32.613656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:32.613656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:32.613656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:32.613656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:32.613656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:32.613656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:32.613656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:32.613656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:32.613656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:32.613656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:32.613656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:32.613656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:32.613656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:32.613656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:32.613656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:32.613656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:32.613656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:16:32.614680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:16:32.614680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:32.614680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:16:32.621378 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 615/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:32.621526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5418bbc-286b-4f33-addb-e822738ecb43 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:32.739344 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5418bbc-286b-4f33-addb-e822738ecb43 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:32.749887 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:32.779064 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82265fc5-2308-4e51-88a1-d8f1b035cb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:32.779603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82265fc5-2308-4e51-88a1-d8f1b035cb95 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:32.779888 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82265fc5-2308-4e51-88a1-d8f1b035cb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:32.780184 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82265fc5-2308-4e51-88a1-d8f1b035cb95 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:32.780924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82265fc5-2308-4e51-88a1-d8f1b035cb95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:32.781338 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82265fc5-2308-4e51-88a1-d8f1b035cb95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:32.782943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82265fc5-2308-4e51-88a1-d8f1b035cb95 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:32.797581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 623/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:16:32.803566 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8576b-d123-4152-8720-d6c1c3eef1a9 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:32.917252 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8576b-d123-4152-8720-d6c1c3eef1a9 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:32.956932 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d160bf-d5ce-4ff2-9f60-32fd1e50fc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:32.957648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d160bf-d5ce-4ff2-9f60-32fd1e50fc97 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:32.957944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d160bf-d5ce-4ff2-9f60-32fd1e50fc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:32.958200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d160bf-d5ce-4ff2-9f60-32fd1e50fc97 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:32.961492 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:32.967961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d160bf-d5ce-4ff2-9f60-32fd1e50fc97 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:32.969815 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d160bf-d5ce-4ff2-9f60-32fd1e50fc97 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:32.983361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-d4d160bf-d5ce-4ff2-9f60-32fd1e50fc97 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 19:16:32.983361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:32.983361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:32.983361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:32.983361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:32.983361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:32.983361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:32.983361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:32.983361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:32.983361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:32.983361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:32.983361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:32.983361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:16:32.983361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:16:32.983361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:16:32.983361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:32.984618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:32.985569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:32.985569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:32.985569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:16:32.985569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:16:32.985569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:16:32.985569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:16:32.985569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:16:32.985569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:16:32.985569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 19:16:32.985569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:16:32.985569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 616/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 19:16:32.990962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82265fc5-2308-4e51-88a1-d8f1b035cb95 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:33.099821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82265fc5-2308-4e51-88a1-d8f1b035cb95 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:33.140171 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18377488-c674-4278-b64e-b90a4e4b976b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:33.140889 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18377488-c674-4278-b64e-b90a4e4b976b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:33.141211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18377488-c674-4278-b64e-b90a4e4b976b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:33.141523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18377488-c674-4278-b64e-b90a4e4b976b None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:33.142438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18377488-c674-4278-b64e-b90a4e4b976b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:33.142913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18377488-c674-4278-b64e-b90a4e4b976b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:33.144988 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18377488-c674-4278-b64e-b90a4e4b976b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:33.169099 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 624/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Sun Aug 7 19:16:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:33.175001 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d160bf-d5ce-4ff2-9f60-32fd1e50fc97 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:33.284023 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d160bf-d5ce-4ff2-9f60-32fd1e50fc97 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:33.322476 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b64ba-897a-4876-ace2-b698d99f3786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:33.323214 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b64ba-897a-4876-ace2-b698d99f3786 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:33.323485 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b64ba-897a-4876-ace2-b698d99f3786 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:33.323819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265b64ba-897a-4876-ace2-b698d99f3786 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:33.393223 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:33.401569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b64ba-897a-4876-ace2-b698d99f3786 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:33.403305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265b64ba-897a-4876-ace2-b698d99f3786 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:33.527839 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 617/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:16:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:16:33.533940 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18377488-c674-4278-b64e-b90a4e4b976b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:33.565740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:33.642716 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18377488-c674-4278-b64e-b90a4e4b976b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:33.682025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b134c-964d-4ade-954b-7b601ebb0a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:33.682760 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b134c-964d-4ade-954b-7b601ebb0a46 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:33.683034 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b134c-964d-4ade-954b-7b601ebb0a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:33.683335 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b134c-964d-4ade-954b-7b601ebb0a46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:33.695685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b134c-964d-4ade-954b-7b601ebb0a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:33.697221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b134c-964d-4ade-954b-7b601ebb0a46 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:33.716213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-2b9b134c-964d-4ade-954b-7b601ebb0a46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:33.716213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:33.716213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:33.716213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:33.716213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:33.716213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:33.716213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:33.716213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:33.716213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:33.716213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:33.716213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:33.716213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:33.716213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:16:33.716213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:16:33.716213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:16:33.716213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:16:33.717640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:33.717640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:33.717640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:33.717640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:33.717640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:33.717640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:33.717640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:33.717640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:33.717640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:33.717640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:33.717640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:33.717640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:33.717640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:33.717640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:33.717640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:33.717640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:33.717640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:16:33.718738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:16:33.718738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:33.718738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:33.718738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 625/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:33.724196 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b64ba-897a-4876-ace2-b698d99f3786 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:33.842575 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:33.849406 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265b64ba-897a-4876-ace2-b698d99f3786 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:33.895843 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e788515-d1b0-4377-a714-c7f4c5ceeb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:33.896486 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e788515-d1b0-4377-a714-c7f4c5ceeb07 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:33.896773 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e788515-d1b0-4377-a714-c7f4c5ceeb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:33.897023 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e788515-d1b0-4377-a714-c7f4c5ceeb07 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:33.897673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e788515-d1b0-4377-a714-c7f4c5ceeb07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:33.897977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e788515-d1b0-4377-a714-c7f4c5ceeb07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:33.899528 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e788515-d1b0-4377-a714-c7f4c5ceeb07 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:33.912616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 618/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:33.919152 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b134c-964d-4ade-954b-7b601ebb0a46 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:34.017084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:34.042501 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b134c-964d-4ade-954b-7b601ebb0a46 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:34.084685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb9723-210d-4871-944a-55450af79457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:34.085408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb9723-210d-4871-944a-55450af79457 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:34.085687 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb9723-210d-4871-944a-55450af79457 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:34.085981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb9723-210d-4871-944a-55450af79457 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:34.093389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb9723-210d-4871-944a-55450af79457 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:34.094878 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb9723-210d-4871-944a-55450af79457 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:34.106638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-45bb9723-210d-4871-944a-55450af79457 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:16:34.106638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:34.106638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:34.106638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:34.106638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:34.106638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:34.106638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:34.106638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:34.106638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:34.106638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:34.106638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:34.106638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:34.106638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:16:34.106638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:16:34.106638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:16:34.107688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:16:34.107688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:34.107688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:34.107688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:34.107688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:34.107688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:34.107688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:34.107688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:34.107688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:34.107688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:34.107688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:34.107688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:34.107688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:34.107688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:34.107688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:34.107688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:34.107688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:34.108631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:16:34.108631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:16:34.108631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:16:34.108631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:16:34.108631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:16:34.108631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:16:34.108631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:16:34.108631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:34.108631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 626/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:16:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:34.114023 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e788515-d1b0-4377-a714-c7f4c5ceeb07 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:34.229849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e788515-d1b0-4377-a714-c7f4c5ceeb07 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:34.265696 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a64af-7082-4d86-a3cc-b252a6b15b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:34.266354 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a64af-7082-4d86-a3cc-b252a6b15b54 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:34.266681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a64af-7082-4d86-a3cc-b252a6b15b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:34.266956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a64af-7082-4d86-a3cc-b252a6b15b54 None admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:34.267691 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a64af-7082-4d86-a3cc-b252a6b15b54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:34.268020 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a64af-7082-4d86-a3cc-b252a6b15b54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:34.269679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a64af-7082-4d86-a3cc-b252a6b15b54 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:34.288848 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 619/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:16:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:16:34.290193 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb9723-210d-4871-944a-55450af79457 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:34.397439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb9723-210d-4871-944a-55450af79457 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:34.434178 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e09c3-8449-4a0a-a755-831bd3230266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:34.434809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e09c3-8449-4a0a-a755-831bd3230266 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:34.435089 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e09c3-8449-4a0a-a755-831bd3230266 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:34.435360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e09c3-8449-4a0a-a755-831bd3230266 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:34.436146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e09c3-8449-4a0a-a755-831bd3230266 None admin] RBAC: Adding query filter params (role.id=43aa4a9830054a09baa1f0aecfd125e2, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=143c30b4201645279564e72ab3288adc) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:34.436453 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e09c3-8449-4a0a-a755-831bd3230266 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:34.437897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e09c3-8449-4a0a-a755-831bd3230266 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:34.443945 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 627/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:16:34 2022] GET /identity/v3/role_assignments?user.id=143c30b4201645279564e72ab3288adc&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=43aa4a9830054a09baa1f0aecfd125e2 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:36.662874 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769a88d-3262-4c71-9ff6-e8b11eff3a9d None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:36.663466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769a88d-3262-4c71-9ff6-e8b11eff3a9d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:36.663466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769a88d-3262-4c71-9ff6-e8b11eff3a9d None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:36.664802 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 620/1247] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:16:36.670357 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdabe8-ace6-4a49-88c8-55d076913514 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:36.671297 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdabe8-ace6-4a49-88c8-55d076913514 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:36.671297 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bdabe8-ace6-4a49-88c8-55d076913514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:36.677298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:36.692844 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-d5bdabe8-ace6-4a49-88c8-55d076913514 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:16:36.694071 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-d5bdabe8-ace6-4a49-88c8-55d076913514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:16:36.768482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 628/1248] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:16:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:16:36.795190 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769a88d-3262-4c71-9ff6-e8b11eff3a9d None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:36.818085 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:36.910275 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769a88d-3262-4c71-9ff6-e8b11eff3a9d None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:36.947943 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c33d3a-2c50-4d36-aa92-d10df7fade85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:36.948611 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c33d3a-2c50-4d36-aa92-d10df7fade85 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:36.948913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c33d3a-2c50-4d36-aa92-d10df7fade85 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:36.949165 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c33d3a-2c50-4d36-aa92-d10df7fade85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:36.960674 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c33d3a-2c50-4d36-aa92-d10df7fade85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:16:36.966727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-44c33d3a-2c50-4d36-aa92-d10df7fade85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:16:36.966727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:36.966727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:36.966727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:36.966727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:36.966727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:36.966727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:36.966727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:36.966727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:36.966727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:36.966727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:36.966727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:36.966727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:16:36.966727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:16:36.966727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:16:36.966727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:16:36.967754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:16:36.967754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:16:36.967754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:16:36.967754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:16:36.967754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:36.967754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:36.967754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:36.967754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:36.967754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:36.967754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:36.967754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:36.967754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:36.967754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:36.967754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:36.967754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:36.967754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:36.968859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:36.968859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:36.968859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:36.968859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:36.968859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:16:36.968859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:16:36.968859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:16:36.968859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:16:36.968859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:16:36.968859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:16:36.968859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:16:36.968859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:16:36.968859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 621/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:16:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:36.975881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bdabe8-ace6-4a49-88c8-55d076913514 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:37.088272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bdabe8-ace6-4a49-88c8-55d076913514 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:37.119730 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:37.125331 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5fddf-7483-45c9-b4d5-96e964d498da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:37.126085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5fddf-7483-45c9-b4d5-96e964d498da None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:37.126618 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5fddf-7483-45c9-b4d5-96e964d498da None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:37.126890 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5fddf-7483-45c9-b4d5-96e964d498da None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:37.127579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5fddf-7483-45c9-b4d5-96e964d498da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:37.127879 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5fddf-7483-45c9-b4d5-96e964d498da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:37.129337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5fddf-7483-45c9-b4d5-96e964d498da None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:37.137683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 629/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:16:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:37.145727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c33d3a-2c50-4d36-aa92-d10df7fade85 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:37.245072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:37.270522 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c33d3a-2c50-4d36-aa92-d10df7fade85 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:37.309471 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf314b0-1ebe-4b56-95d4-459a567d113f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:37.310132 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf314b0-1ebe-4b56-95d4-459a567d113f None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:37.310455 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf314b0-1ebe-4b56-95d4-459a567d113f None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:37.310738 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf314b0-1ebe-4b56-95d4-459a567d113f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:37.321927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf314b0-1ebe-4b56-95d4-459a567d113f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:37.323482 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf314b0-1ebe-4b56-95d4-459a567d113f None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:37.341746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-8bf314b0-1ebe-4b56-95d4-459a567d113f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:37.341746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:37.341746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:37.341746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:37.341746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:37.341746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:37.341746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:37.341746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:37.341746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:37.341746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:37.341746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:37.341746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:37.341746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:16:37.341746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:16:37.341746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:16:37.341746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:16:37.342855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:37.342855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:37.342855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:37.342855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:37.342855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:37.342855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:37.342855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:37.342855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:37.342855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:37.342855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:37.342855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:37.342855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:37.342855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:37.342855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:37.342855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:37.342855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:37.342855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:16:37.343826 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:16:37.343826 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:37.343826 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:16:37.343826 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 622/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:37.349664 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5fddf-7483-45c9-b4d5-96e964d498da None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:37.458504 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5fddf-7483-45c9-b4d5-96e964d498da None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:37.497918 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f72b5b-e583-4c78-a04a-91cf374bbe7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:37.498369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f72b5b-e583-4c78-a04a-91cf374bbe7e None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:37.498647 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f72b5b-e583-4c78-a04a-91cf374bbe7e None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:37.498911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f72b5b-e583-4c78-a04a-91cf374bbe7e None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:37.499553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f72b5b-e583-4c78-a04a-91cf374bbe7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:37.499857 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f72b5b-e583-4c78-a04a-91cf374bbe7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:37.501258 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f72b5b-e583-4c78-a04a-91cf374bbe7e None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:37.518074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 630/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:37.525934 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf314b0-1ebe-4b56-95d4-459a567d113f None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:37.634314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf314b0-1ebe-4b56-95d4-459a567d113f None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:37.665031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:37.674097 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd8ff3-806d-4382-be1e-7b7f01edfcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:37.674785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd8ff3-806d-4382-be1e-7b7f01edfcc3 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:37.675062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd8ff3-806d-4382-be1e-7b7f01edfcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:37.675305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd8ff3-806d-4382-be1e-7b7f01edfcc3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:37.683481 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd8ff3-806d-4382-be1e-7b7f01edfcc3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:37.684950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd8ff3-806d-4382-be1e-7b7f01edfcc3 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:37.697331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-5cdd8ff3-806d-4382-be1e-7b7f01edfcc3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 19:16:37.697331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:37.697331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:37.697331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:37.697331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:37.697331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:37.697331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:37.697331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:37.697331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:37.697331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:37.697331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:37.697331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:37.697331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:16:37.697331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:16:37.697331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:16:37.697331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:37.698380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:37.699393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:37.699393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:37.699393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:16:37.699393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:16:37.699393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:16:37.699393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:16:37.699393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:16:37.699393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:16:37.699393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 19:16:37.699393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:16:37.699393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 623/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:37.704855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f72b5b-e583-4c78-a04a-91cf374bbe7e None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:37.771858 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:37.818723 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f72b5b-e583-4c78-a04a-91cf374bbe7e None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:37.853586 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073820f1-0f6f-4198-b48d-a31638d2fd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:37.854246 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073820f1-0f6f-4198-b48d-a31638d2fd22 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:37.854529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073820f1-0f6f-4198-b48d-a31638d2fd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:37.854801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073820f1-0f6f-4198-b48d-a31638d2fd22 None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:37.855559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073820f1-0f6f-4198-b48d-a31638d2fd22 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:37.855862 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073820f1-0f6f-4198-b48d-a31638d2fd22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:37.857344 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073820f1-0f6f-4198-b48d-a31638d2fd22 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:37.875758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 631/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Sun Aug 7 19:16:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:37.882421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd8ff3-806d-4382-be1e-7b7f01edfcc3 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:37.989267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd8ff3-806d-4382-be1e-7b7f01edfcc3 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:38.027524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97832db3-0460-4c32-9cfb-82ee017942c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:38.028113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97832db3-0460-4c32-9cfb-82ee017942c7 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:38.028429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97832db3-0460-4c32-9cfb-82ee017942c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:38.028703 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97832db3-0460-4c32-9cfb-82ee017942c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:38.089759 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:38.103094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97832db3-0460-4c32-9cfb-82ee017942c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:38.104762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97832db3-0460-4c32-9cfb-82ee017942c7 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:38.240420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 624/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:16:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:16:38.241802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073820f1-0f6f-4198-b48d-a31638d2fd22 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:38.357302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073820f1-0f6f-4198-b48d-a31638d2fd22 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:38.371204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:38.393988 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14cdebe-b5ac-4592-b457-eff143a8ffa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:38.394626 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cdebe-b5ac-4592-b457-eff143a8ffa5 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:38.394900 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cdebe-b5ac-4592-b457-eff143a8ffa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:38.395165 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cdebe-b5ac-4592-b457-eff143a8ffa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:38.406103 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14cdebe-b5ac-4592-b457-eff143a8ffa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:38.407699 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14cdebe-b5ac-4592-b457-eff143a8ffa5 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:38.422742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-f14cdebe-b5ac-4592-b457-eff143a8ffa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:38.422742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:38.422742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:38.422742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:38.422742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:38.422742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:38.422742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:38.422742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:38.422742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:38.422742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:38.422742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:38.422742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:38.422742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:16:38.422742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:16:38.422742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:16:38.422742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:16:38.423820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:38.423820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:38.423820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:38.423820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:38.423820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:38.423820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:38.423820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:38.423820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:38.423820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:38.423820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:38.423820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:38.423820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:38.423820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:38.423820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:38.423820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:38.423820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:38.423820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:16:38.429280 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:16:38.429280 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:38.429280 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:38.429280 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 632/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:38.429758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97832db3-0460-4c32-9cfb-82ee017942c7 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:38.522946 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:38.537441 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97832db3-0460-4c32-9cfb-82ee017942c7 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:38.581430 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8ad6d-5d30-4346-a434-7acc989a6aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:38.582088 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8ad6d-5d30-4346-a434-7acc989a6aea None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:38.582363 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8ad6d-5d30-4346-a434-7acc989a6aea None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:38.582672 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8ad6d-5d30-4346-a434-7acc989a6aea None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:38.583324 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f8ad6d-5d30-4346-a434-7acc989a6aea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:38.583632 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f8ad6d-5d30-4346-a434-7acc989a6aea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:38.585165 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f8ad6d-5d30-4346-a434-7acc989a6aea None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:38.600432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 625/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:16:38.607220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14cdebe-b5ac-4592-b457-eff143a8ffa5 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:38.726979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14cdebe-b5ac-4592-b457-eff143a8ffa5 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:38.770025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02189059-cc9d-4c3d-83c5-9837643dbb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:38.770972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02189059-cc9d-4c3d-83c5-9837643dbb94 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:38.771376 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02189059-cc9d-4c3d-83c5-9837643dbb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:38.771733 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02189059-cc9d-4c3d-83c5-9837643dbb94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:38.780399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02189059-cc9d-4c3d-83c5-9837643dbb94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:38.781825 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02189059-cc9d-4c3d-83c5-9837643dbb94 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:38.794255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-02189059-cc9d-4c3d-83c5-9837643dbb94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:16:38.794255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:38.794255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:38.794255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:38.794255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:38.794255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:38.794255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:38.794255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:38.794255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:38.794255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:38.794255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:38.794255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:38.794255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:16:38.794255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:16:38.794255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:16:38.795188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:16:38.795188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:38.795188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:38.795188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:38.795188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:38.795188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:38.795188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:38.795188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:38.795188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:38.795188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:38.795188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:38.795188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:38.795188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:38.795188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:38.795188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:38.795188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:38.795188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:38.796257 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:16:38.796257 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:16:38.796257 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:16:38.796257 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:16:38.796257 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:16:38.796257 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:16:38.796257 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:16:38.796257 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:38.800358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 633/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:16:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:38.801684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8ad6d-5d30-4346-a434-7acc989a6aea None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:38.910291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8ad6d-5d30-4346-a434-7acc989a6aea None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:38.945670 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21de07d-c140-4bbe-b8c5-7c15ab63fe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:38.946351 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21de07d-c140-4bbe-b8c5-7c15ab63fe4e None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:38.946568 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21de07d-c140-4bbe-b8c5-7c15ab63fe4e None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:38.946868 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21de07d-c140-4bbe-b8c5-7c15ab63fe4e None admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:38.947660 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21de07d-c140-4bbe-b8c5-7c15ab63fe4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:38.948003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21de07d-c140-4bbe-b8c5-7c15ab63fe4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:38.949476 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21de07d-c140-4bbe-b8c5-7c15ab63fe4e None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:38.962003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 626/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:16:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:38.968304 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02189059-cc9d-4c3d-83c5-9837643dbb94 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:38.995126 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:39.079851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02189059-cc9d-4c3d-83c5-9837643dbb94 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:39.117329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49138f59-cff0-41a4-bd4a-6ddd549ab89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:39.117946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49138f59-cff0-41a4-bd4a-6ddd549ab89d None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:39.118227 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49138f59-cff0-41a4-bd4a-6ddd549ab89d None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:39.118499 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49138f59-cff0-41a4-bd4a-6ddd549ab89d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:39.119233 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49138f59-cff0-41a4-bd4a-6ddd549ab89d None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=143c30b4201645279564e72ab3288adc) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:39.119592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49138f59-cff0-41a4-bd4a-6ddd549ab89d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:39.121128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49138f59-cff0-41a4-bd4a-6ddd549ab89d None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:39.127068 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 634/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:16:38 2022] GET /identity/v3/role_assignments?user.id=143c30b4201645279564e72ab3288adc&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=4da42f08d107448fb877b42b61bc74ea => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:41.167129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed650c7b-cab3-40f7-92c1-8ac3ecf564b0 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:41.167721 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed650c7b-cab3-40f7-92c1-8ac3ecf564b0 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:41.167721 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed650c7b-cab3-40f7-92c1-8ac3ecf564b0 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:41.169037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 627/1261] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:16:41.176539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886583f2-2cb8-4a56-805e-3c4b79dfa745 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:41.176958 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886583f2-2cb8-4a56-805e-3c4b79dfa745 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:41.177071 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886583f2-2cb8-4a56-805e-3c4b79dfa745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:41.196344 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-886583f2-2cb8-4a56-805e-3c4b79dfa745 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:16:41.197546 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-886583f2-2cb8-4a56-805e-3c4b79dfa745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:16:41.268228 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 635/1262] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:16:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:16:41.290566 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed650c7b-cab3-40f7-92c1-8ac3ecf564b0 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:41.302026 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:41.393302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed650c7b-cab3-40f7-92c1-8ac3ecf564b0 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:41.427052 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8725eda-baf9-4dde-bfe8-66847abe7014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:41.427665 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8725eda-baf9-4dde-bfe8-66847abe7014 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:41.427937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8725eda-baf9-4dde-bfe8-66847abe7014 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:41.428240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8725eda-baf9-4dde-bfe8-66847abe7014 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:41.439107 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8725eda-baf9-4dde-bfe8-66847abe7014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:16:41.443714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-e8725eda-baf9-4dde-bfe8-66847abe7014 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:16:41.443714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:41.443714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:41.443714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:41.443714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:41.443714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:41.443714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:41.443714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:41.443714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:41.443714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:41.443714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:41.443714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:41.443714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:16:41.443714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:16:41.443714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:16:41.443714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:16:41.444904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:16:41.444904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:16:41.444904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:16:41.444904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:16:41.444904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:41.444904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:41.444904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:41.444904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:41.444904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:41.444904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:41.444904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:41.444904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:41.444904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:41.444904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:41.444904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:41.444904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:41.446037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:41.446037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:41.446037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:41.446037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:41.446037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:16:41.446037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:16:41.446037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:16:41.446037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:16:41.446037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:16:41.446037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:16:41.446037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:16:41.446037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:16:41.446037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 628/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:16:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:41.451122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886583f2-2cb8-4a56-805e-3c4b79dfa745 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:41.469574 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:41.551283 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886583f2-2cb8-4a56-805e-3c4b79dfa745 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:41.585876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41243430-b5cb-4999-b642-d997aff9e7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:41.586480 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41243430-b5cb-4999-b642-d997aff9e7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:41.586751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41243430-b5cb-4999-b642-d997aff9e7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:41.587017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41243430-b5cb-4999-b642-d997aff9e7d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:41.587658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41243430-b5cb-4999-b642-d997aff9e7d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:41.587955 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41243430-b5cb-4999-b642-d997aff9e7d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:41.589425 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41243430-b5cb-4999-b642-d997aff9e7d6 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:41.597493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 636/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:16:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:41.603601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8725eda-baf9-4dde-bfe8-66847abe7014 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:41.671633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:41.707237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8725eda-baf9-4dde-bfe8-66847abe7014 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:41.747541 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6ab89-f49e-45d1-b8d0-91f1d926a5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:41.748284 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6ab89-f49e-45d1-b8d0-91f1d926a5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:41.748610 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6ab89-f49e-45d1-b8d0-91f1d926a5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:41.748911 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6ab89-f49e-45d1-b8d0-91f1d926a5c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:41.759970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6ab89-f49e-45d1-b8d0-91f1d926a5c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:41.761496 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6ab89-f49e-45d1-b8d0-91f1d926a5c6 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:41.776077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-35e6ab89-f49e-45d1-b8d0-91f1d926a5c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:41.776077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:41.776077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:41.776077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:41.776077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:41.776077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:41.776077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:41.776077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:41.776077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:41.776077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:41.776077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:41.776077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:41.776077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:16:41.776077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:16:41.776077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:16:41.776077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:16:41.777200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:41.777200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:41.777200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:41.777200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:41.777200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:41.777200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:41.777200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:41.777200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:41.777200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:41.777200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:41.777200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:41.777200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:41.777200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:41.777200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:41.777200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:41.777200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:41.777200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:16:41.778376 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:16:41.778376 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:41.778376 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:16:41.782112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 629/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:41.783278 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41243430-b5cb-4999-b642-d997aff9e7d6 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:41.866694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:41.891002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41243430-b5cb-4999-b642-d997aff9e7d6 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:41.928904 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14198b8c-f1d1-4ea7-89a4-3468bd84140b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:41.929547 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14198b8c-f1d1-4ea7-89a4-3468bd84140b None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:41.929843 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14198b8c-f1d1-4ea7-89a4-3468bd84140b None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:41.930133 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14198b8c-f1d1-4ea7-89a4-3468bd84140b None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:41.930815 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14198b8c-f1d1-4ea7-89a4-3468bd84140b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:41.931134 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14198b8c-f1d1-4ea7-89a4-3468bd84140b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:41.932967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14198b8c-f1d1-4ea7-89a4-3468bd84140b None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:41.945621 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 637/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:41.952147 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6ab89-f49e-45d1-b8d0-91f1d926a5c6 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:42.065408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6ab89-f49e-45d1-b8d0-91f1d926a5c6 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:42.085484 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:42.099958 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8dd3a-e6a3-4ba3-8f74-765fb68fabf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:42.100670 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8dd3a-e6a3-4ba3-8f74-765fb68fabf3 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:42.100956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8dd3a-e6a3-4ba3-8f74-765fb68fabf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:42.101233 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8dd3a-e6a3-4ba3-8f74-765fb68fabf3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:42.108946 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8dd3a-e6a3-4ba3-8f74-765fb68fabf3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:42.110449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8dd3a-e6a3-4ba3-8f74-765fb68fabf3 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:42.122104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-35d8dd3a-e6a3-4ba3-8f74-765fb68fabf3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 19:16:42.122104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:42.122104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:42.122104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:42.122104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:42.122104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:42.122104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:42.122104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:42.122104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:42.122104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:42.122104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:42.122104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:42.122104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:16:42.122104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:16:42.122104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:16:42.122104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:42.123271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:42.124702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:42.124702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:42.124702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:16:42.124702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:16:42.124702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:16:42.124702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:16:42.124702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:16:42.124702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:16:42.124702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 19:16:42.124702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:16:42.124702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 630/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:42.129351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14198b8c-f1d1-4ea7-89a4-3468bd84140b None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:42.235038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14198b8c-f1d1-4ea7-89a4-3468bd84140b None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:42.281282 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbab54f-dc83-4169-ac6e-90232ac7ac9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:42.281890 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbab54f-dc83-4169-ac6e-90232ac7ac9f None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:42.282165 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbab54f-dc83-4169-ac6e-90232ac7ac9f None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:42.282438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbab54f-dc83-4169-ac6e-90232ac7ac9f None admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:42.283160 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbab54f-dc83-4169-ac6e-90232ac7ac9f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:42.283464 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbab54f-dc83-4169-ac6e-90232ac7ac9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:42.284919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbab54f-dc83-4169-ac6e-90232ac7ac9f None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:42.293843 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:42.303356 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 638/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Sun Aug 7 19:16:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:42.309808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8dd3a-e6a3-4ba3-8f74-765fb68fabf3 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:42.410830 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8dd3a-e6a3-4ba3-8f74-765fb68fabf3 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:42.446146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f559d-7de2-4da1-920a-5b23394d0b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:42.446774 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f559d-7de2-4da1-920a-5b23394d0b81 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:42.447054 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f559d-7de2-4da1-920a-5b23394d0b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:42.447327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f559d-7de2-4da1-920a-5b23394d0b81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:42.458512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f559d-7de2-4da1-920a-5b23394d0b81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:42.459966 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f559d-7de2-4da1-920a-5b23394d0b81 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:42.474185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-cc4f559d-7de2-4da1-920a-5b23394d0b81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:42.474185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:42.474185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:42.474185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:42.474185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:42.474185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:42.474185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:42.474185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:42.474185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:42.474185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:42.474185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:42.474185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:42.474185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:16:42.474185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:16:42.474185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:16:42.474185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:16:42.475262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:42.475262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:42.475262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:42.475262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:42.475262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:42.475262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:42.475262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:42.475262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:42.475262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:42.475262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:42.475262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:42.475262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:42.475262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:42.475262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:42.475262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:42.475262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:42.475262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:16:42.476394 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:16:42.476394 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:42.476394 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:16:42.476394 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 631/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:42.481530 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbab54f-dc83-4169-ac6e-90232ac7ac9f None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:42.586919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbab54f-dc83-4169-ac6e-90232ac7ac9f None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:42.623698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37317c-6c22-4983-8f45-e0922aaafdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:42.624351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37317c-6c22-4983-8f45-e0922aaafdd5 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:42.624632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37317c-6c22-4983-8f45-e0922aaafdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:42.624907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37317c-6c22-4983-8f45-e0922aaafdd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:42.625556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37317c-6c22-4983-8f45-e0922aaafdd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:42.625857 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37317c-6c22-4983-8f45-e0922aaafdd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:42.627246 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f37317c-6c22-4983-8f45-e0922aaafdd5 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:42.639216 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 639/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:42.645502 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f559d-7de2-4da1-920a-5b23394d0b81 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:42.678742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:42.757888 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f559d-7de2-4da1-920a-5b23394d0b81 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:42.793946 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136f384-e5c7-49c7-85c4-ef3cf0c90f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:42.794586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136f384-e5c7-49c7-85c4-ef3cf0c90f4e None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:42.794862 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136f384-e5c7-49c7-85c4-ef3cf0c90f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:42.795130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136f384-e5c7-49c7-85c4-ef3cf0c90f4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:42.802625 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1136f384-e5c7-49c7-85c4-ef3cf0c90f4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:42.804158 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1136f384-e5c7-49c7-85c4-ef3cf0c90f4e None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:42.816449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-1136f384-e5c7-49c7-85c4-ef3cf0c90f4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:16:42.816449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:42.816449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:42.816449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:42.816449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:42.816449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:42.816449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:42.816449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:42.816449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:42.816449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:42.816449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:42.816449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:42.816449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:16:42.816449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:16:42.816449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:16:42.817618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:16:42.817618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:42.817618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:42.817618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:42.817618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:42.817618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:42.817618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:42.817618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:42.817618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:42.817618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:42.817618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:42.817618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:42.817618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:42.817618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:42.817618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:42.817618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:42.817618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:42.818530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:16:42.818530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:16:42.818530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:16:42.818530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:16:42.818530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:16:42.818530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:16:42.818530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:16:42.818530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:16:42.818530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 632/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:16:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:42.823940 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37317c-6c22-4983-8f45-e0922aaafdd5 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:42.884768 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:42.930558 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f37317c-6c22-4983-8f45-e0922aaafdd5 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:42.965456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b26ba-e8ce-4ffd-9ac8-b4f5736f5aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:42.966063 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b26ba-e8ce-4ffd-9ac8-b4f5736f5aca None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:42.966340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b26ba-e8ce-4ffd-9ac8-b4f5736f5aca None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:42.966614 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b26ba-e8ce-4ffd-9ac8-b4f5736f5aca None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:42.967305 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b26ba-e8ce-4ffd-9ac8-b4f5736f5aca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:42.967607 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b26ba-e8ce-4ffd-9ac8-b4f5736f5aca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:42.969066 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b26ba-e8ce-4ffd-9ac8-b4f5736f5aca None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:42.981260 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 640/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:16:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:42.987883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136f384-e5c7-49c7-85c4-ef3cf0c90f4e None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:43.082931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:43.093252 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1136f384-e5c7-49c7-85c4-ef3cf0c90f4e None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:43.126563 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0de01-20a5-4734-a5bb-a17bbbd96c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:43.127219 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0de01-20a5-4734-a5bb-a17bbbd96c27 None admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:43.127498 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0de01-20a5-4734-a5bb-a17bbbd96c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:43.127769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0de01-20a5-4734-a5bb-a17bbbd96c27 None admin] PATH_INFO: `/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/143c30b4201645279564e72ab3288adc/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:43.129679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0de01-20a5-4734-a5bb-a17bbbd96c27 None admin] RBAC: Authorizing `identity:create_grant(project_id=37703d50cd7e4d9bbdc797a7a1466b66, user_id=143c30b4201645279564e72ab3288adc, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:43.131115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0de01-20a5-4734-a5bb-a17bbbd96c27 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:43.145146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 633/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Sun Aug 7 19:16:42 2022] PUT /identity/v3/projects/37703d50cd7e4d9bbdc797a7a1466b66/users/143c30b4201645279564e72ab3288adc/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:16:45.112289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40186aed-8766-4299-b28b-88ff1082e640 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:45.112844 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40186aed-8766-4299-b28b-88ff1082e640 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:45.112844 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40186aed-8766-4299-b28b-88ff1082e640 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:45.114190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 641/1274] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:16:45.119645 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131d4a2-468f-471b-a610-944372729c76 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:45.120100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131d4a2-468f-471b-a610-944372729c76 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:45.120239 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131d4a2-468f-471b-a610-944372729c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:45.139402 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-e131d4a2-468f-471b-a610-944372729c76 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:16:45.140738 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e131d4a2-468f-471b-a610-944372729c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:16:45.213952 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 634/1275] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:16:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:16:45.237472 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40186aed-8766-4299-b28b-88ff1082e640 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:45.347363 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40186aed-8766-4299-b28b-88ff1082e640 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:45.366850 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:45.387010 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b100b2d-87c8-418f-ab30-090907b3d8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:45.387661 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b100b2d-87c8-418f-ab30-090907b3d8ba None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:45.387937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b100b2d-87c8-418f-ab30-090907b3d8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:45.388258 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b100b2d-87c8-418f-ab30-090907b3d8ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:45.400158 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b100b2d-87c8-418f-ab30-090907b3d8ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 19:16:45.404956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-5b100b2d-87c8-418f-ab30-090907b3d8ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:16:45.404956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:45.404956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:45.404956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:45.404956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:45.404956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:45.404956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:45.404956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:45.404956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:45.404956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:45.404956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:45.404956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:45.404956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 19:16:45.404956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 19:16:45.404956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 19:16:45.404956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 19:16:45.406056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 19:16:45.406056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 19:16:45.406056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 19:16:45.406056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 19:16:45.406056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:45.406056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:45.406056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:45.406056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:45.406056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:45.406056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:45.406056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:45.406056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:45.406056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:45.406056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:45.406056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:45.406056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:45.406947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:45.406947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:45.406947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:45.406947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:45.406947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 19:16:45.406947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 19:16:45.406947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 19:16:45.406947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 19:16:45.406947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 19:16:45.406947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 19:16:45.406947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 19:16:45.406947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:45.406947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 642/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Sun Aug 7 19:16:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:45.412702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131d4a2-468f-471b-a610-944372729c76 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:45.497897 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:45.518457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131d4a2-468f-471b-a610-944372729c76 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:45.555984 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc77c85-90e7-4957-949d-edefca2277d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:45.557022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc77c85-90e7-4957-949d-edefca2277d7 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:45.557304 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc77c85-90e7-4957-949d-edefca2277d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:45.557575 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc77c85-90e7-4957-949d-edefca2277d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:45.558242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc77c85-90e7-4957-949d-edefca2277d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:45.558545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc77c85-90e7-4957-949d-edefca2277d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:45.559945 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc77c85-90e7-4957-949d-edefca2277d7 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:45.568587 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 635/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Sun Aug 7 19:16:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:45.574715 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b100b2d-87c8-418f-ab30-090907b3d8ba None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:45.680173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b100b2d-87c8-418f-ab30-090907b3d8ba None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:45.716995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd38b5-def0-459d-8f66-161a80fa94ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:45.717604 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd38b5-def0-459d-8f66-161a80fa94ed None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:45.717882 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd38b5-def0-459d-8f66-161a80fa94ed None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:45.718193 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd38b5-def0-459d-8f66-161a80fa94ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:45.728853 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd38b5-def0-459d-8f66-161a80fa94ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:45.730351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fd38b5-def0-459d-8f66-161a80fa94ed None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:45.746932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-09fd38b5-def0-459d-8f66-161a80fa94ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:45.746932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:45.746932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:45.746932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:45.746932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:45.746932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:45.746932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:45.746932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:45.746932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:45.746932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:45.746932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:45.746932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:45.746932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:16:45.746932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:16:45.746932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:16:45.746932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:16:45.748190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:45.748190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:45.748190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:45.748190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:45.748190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:45.748190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:45.748190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:45.748190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:45.748190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:45.748190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:45.748190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:45.748190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:45.748190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:45.748190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:45.748190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:45.748190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:45.748190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:16:45.749097 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:16:45.749097 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:45.749097 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:45.749097 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 643/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:45.754195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc77c85-90e7-4957-949d-edefca2277d7 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:45.867483 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc77c85-90e7-4957-949d-edefca2277d7 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:45.904425 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a026a-d100-40e2-aa20-924832465227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:45.905033 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a026a-d100-40e2-aa20-924832465227 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:45.905313 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a026a-d100-40e2-aa20-924832465227 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:45.905584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a026a-d100-40e2-aa20-924832465227 None admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:45.906228 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a026a-d100-40e2-aa20-924832465227 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:45.906531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a026a-d100-40e2-aa20-924832465227 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:45.907898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a026a-d100-40e2-aa20-924832465227 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:45.920648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 636/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:45.926647 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd38b5-def0-459d-8f66-161a80fa94ed None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:45.959540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:46.036181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd38b5-def0-459d-8f66-161a80fa94ed None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:46.072778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253c967-8caa-43e4-975d-4dd46aa8aaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:46.073416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253c967-8caa-43e4-975d-4dd46aa8aaab None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:46.073691 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253c967-8caa-43e4-975d-4dd46aa8aaab None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:46.073964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253c967-8caa-43e4-975d-4dd46aa8aaab None admin] PATH_INFO: `/v3/users/placement` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:46.081990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8253c967-8caa-43e4-975d-4dd46aa8aaab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:46.083456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8253c967-8caa-43e4-975d-4dd46aa8aaab None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:46.094717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-8253c967-8caa-43e4-975d-4dd46aa8aaab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 19:16:46.094717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:46.094717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:46.094717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:46.094717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:46.094717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:46.094717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:46.094717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:46.094717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:46.094717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:46.094717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:46.094717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:46.094717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 19:16:46.094717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 19:16:46.094717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 19:16:46.094717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:46.095937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:46.096995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:46.096995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:46.096995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 19:16:46.096995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 19:16:46.096995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 19:16:46.096995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 19:16:46.096995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 19:16:46.096995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 19:16:46.096995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 19:16:46.096995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:46.096995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 644/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:46.101943 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a026a-d100-40e2-aa20-924832465227 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:46.106399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:46.206701 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a026a-d100-40e2-aa20-924832465227 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:46.242369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6c27f-4297-49d3-822c-b879fb24a1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:46.242885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6c27f-4297-49d3-822c-b879fb24a1fe None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:46.243157 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6c27f-4297-49d3-822c-b879fb24a1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:46.243425 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6c27f-4297-49d3-822c-b879fb24a1fe None admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:46.244175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6c27f-4297-49d3-822c-b879fb24a1fe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:46.244497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6c27f-4297-49d3-822c-b879fb24a1fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:46.245912 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6c27f-4297-49d3-822c-b879fb24a1fe None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:46.263711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 637/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Sun Aug 7 19:16:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:46.269741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253c967-8caa-43e4-975d-4dd46aa8aaab None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:46.373841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:46.385003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253c967-8caa-43e4-975d-4dd46aa8aaab None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:46.418939 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc84cfd-681d-4c25-994b-b58284ce22b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:46.419557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc84cfd-681d-4c25-994b-b58284ce22b6 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:46.419840 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc84cfd-681d-4c25-994b-b58284ce22b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:46.420131 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc84cfd-681d-4c25-994b-b58284ce22b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:46.489657 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc84cfd-681d-4c25-994b-b58284ce22b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:46.491137 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc84cfd-681d-4c25-994b-b58284ce22b6 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:46.583065 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:46.594229 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 645/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Sun Aug 7 19:16:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 19:16:46.600413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6c27f-4297-49d3-822c-b879fb24a1fe None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:46.672043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:46.718268 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6c27f-4297-49d3-822c-b879fb24a1fe None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:46.757348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ba534-b341-4239-874c-123409f715b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:46.757978 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ba534-b341-4239-874c-123409f715b9 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:46.758257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ba534-b341-4239-874c-123409f715b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:46.758526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ba534-b341-4239-874c-123409f715b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:46.769529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ba534-b341-4239-874c-123409f715b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:46.770994 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ba534-b341-4239-874c-123409f715b9 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:46.786918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-384ba534-b341-4239-874c-123409f715b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:46.786918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:46.786918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:46.786918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:46.786918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:46.786918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:46.786918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:46.786918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:46.786918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:46.786918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:46.786918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:46.786918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:46.786918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 19:16:46.786918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 19:16:46.786918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 19:16:46.786918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 19:16:46.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:46.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:46.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:46.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:46.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:46.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:46.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:46.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:46.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:46.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:46.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:46.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:46.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:46.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:46.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:46.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:46.788152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 19:16:46.789065 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 19:16:46.789065 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 19:16:46.789065 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:16:46.789065 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 638/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:16:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:46.794372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc84cfd-681d-4c25-994b-b58284ce22b6 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:46.896756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc84cfd-681d-4c25-994b-b58284ce22b6 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:46.934538 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be3da8-0db7-4889-aebf-b9bda40ca815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:46.935154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be3da8-0db7-4889-aebf-b9bda40ca815 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:46.935429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be3da8-0db7-4889-aebf-b9bda40ca815 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:46.935704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be3da8-0db7-4889-aebf-b9bda40ca815 None admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:46.936386 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be3da8-0db7-4889-aebf-b9bda40ca815 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:46.936688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be3da8-0db7-4889-aebf-b9bda40ca815 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:46.938071 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be3da8-0db7-4889-aebf-b9bda40ca815 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:46.950495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 646/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Sun Aug 7 19:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:46.956641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ba534-b341-4239-874c-123409f715b9 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:47.062611 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ba534-b341-4239-874c-123409f715b9 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:47.077230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:47.099245 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bfaf0-8fd3-4f70-9131-1798a20b8e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:47.099887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bfaf0-8fd3-4f70-9131-1798a20b8e23 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:47.100195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bfaf0-8fd3-4f70-9131-1798a20b8e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:47.100472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bfaf0-8fd3-4f70-9131-1798a20b8e23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:47.107595 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bfaf0-8fd3-4f70-9131-1798a20b8e23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:47.109126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bfaf0-8fd3-4f70-9131-1798a20b8e23 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:47.120536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application [None req-0c0bfaf0-8fd3-4f70-9131-1798a20b8e23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:16:47.120536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:47.120536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:47.120536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:47.120536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:47.120536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:47.120536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:47.120536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:47.120536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:47.120536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:47.120536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:47.120536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:47.120536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 19:16:47.120536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 19:16:47.120536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 19:16:47.121604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 19:16:47.121604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:47.121604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:47.121604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:47.121604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:47.121604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:47.121604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:47.121604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:47.121604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:47.121604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:47.121604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:47.121604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:47.121604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:47.121604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:47.121604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:47.121604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:47.121604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:47.122733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 19:16:47.122733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 19:16:47.122733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 19:16:47.122733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 19:16:47.122733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 19:16:47.122733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 19:16:47.122733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 19:16:47.122733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: ERROR keystone.server.flask.application Aug 07 19:16:47.122733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 639/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Sun Aug 7 19:16:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:47.127867 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be3da8-0db7-4889-aebf-b9bda40ca815 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:47.180025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:47.233738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be3da8-0db7-4889-aebf-b9bda40ca815 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:47.268526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58506f63-d2ae-451f-9bf8-003bb5516097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:47.269136 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58506f63-d2ae-451f-9bf8-003bb5516097 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:47.269412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58506f63-d2ae-451f-9bf8-003bb5516097 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:47.269673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58506f63-d2ae-451f-9bf8-003bb5516097 None admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:47.270361 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58506f63-d2ae-451f-9bf8-003bb5516097 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:47.270936 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58506f63-d2ae-451f-9bf8-003bb5516097 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:47.272107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58506f63-d2ae-451f-9bf8-003bb5516097 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:47.284435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 647/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Sun Aug 7 19:16:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:47.290883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bfaf0-8fd3-4f70-9131-1798a20b8e23 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:47.407932 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bfaf0-8fd3-4f70-9131-1798a20b8e23 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:47.445835 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc967022-175b-4f7d-9e17-0f27297ee618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:47.446349 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc967022-175b-4f7d-9e17-0f27297ee618 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:47.446619 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc967022-175b-4f7d-9e17-0f27297ee618 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:47.446883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc967022-175b-4f7d-9e17-0f27297ee618 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:47.447622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc967022-175b-4f7d-9e17-0f27297ee618 None admin] RBAC: Adding query filter params (role.id=4da42f08d107448fb877b42b61bc74ea, scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66, user.id=143c30b4201645279564e72ab3288adc) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:47.447933 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc967022-175b-4f7d-9e17-0f27297ee618 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:47.449399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc967022-175b-4f7d-9e17-0f27297ee618 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:47.455695 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 640/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Sun Aug 7 19:16:47 2022] GET /identity/v3/role_assignments?user.id=143c30b4201645279564e72ab3288adc&scope.project.id=37703d50cd7e4d9bbdc797a7a1466b66&role.id=4da42f08d107448fb877b42b61bc74ea => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:49.433365 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe201836-69ea-4ec4-bc63-f8d5270df483 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:49.433935 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe201836-69ea-4ec4-bc63-f8d5270df483 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:49.433935 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe201836-69ea-4ec4-bc63-f8d5270df483 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:49.435274 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 648/1288] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:16:49.440431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090453f3-a600-4b88-b4d6-ca0e777a4ff1 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:49.440860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090453f3-a600-4b88-b4d6-ca0e777a4ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:49.440962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090453f3-a600-4b88-b4d6-ca0e777a4ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:49.459231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-090453f3-a600-4b88-b4d6-ca0e777a4ff1 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:16:49.460447 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-090453f3-a600-4b88-b4d6-ca0e777a4ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:16:49.490857 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:49.530166 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 641/1289] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:16:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:16:49.552331 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe201836-69ea-4ec4-bc63-f8d5270df483 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:49.658218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe201836-69ea-4ec4-bc63-f8d5270df483 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:49.670364 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:49.696089 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73ea37-5f8c-4ded-ad84-58eb46aa6b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:49.696669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73ea37-5f8c-4ded-ad84-58eb46aa6b65 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:49.696948 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73ea37-5f8c-4ded-ad84-58eb46aa6b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:49.697221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73ea37-5f8c-4ded-ad84-58eb46aa6b65 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:49.697828 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73ea37-5f8c-4ded-ad84-58eb46aa6b65 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:49.699198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73ea37-5f8c-4ded-ad84-58eb46aa6b65 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:49.709139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-af73ea37-5f8c-4ded-ad84-58eb46aa6b65 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 19:16:49.709139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:49.709139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:16:49.709139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:16:49.709139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:16:49.709139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:16:49.709139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:16:49.709139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:16:49.709139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 19:16:49.709139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:49.709139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:16:49.709139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:49.709139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:49.709139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:49.709139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:49.709139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:49.709139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:49.710429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:49.710429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:49.710429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:49.710429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:49.710429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:49.710429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:49.710429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:16:49.710429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:16:49.710429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:16:49.710429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:16:49.710429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:49.710429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:49.710429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:49.710429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:49.710429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:49.710429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:49.711597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:49.711597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:49.711597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:49.711597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:49.711597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:49.711597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:49.711597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:49.711597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:49.711597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:49.711597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:49.711597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:16:49.711597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:16:49.711597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 19:16:49.711597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:49.711597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 649/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Aug 7 19:16:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:49.717884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090453f3-a600-4b88-b4d6-ca0e777a4ff1 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:49.830782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090453f3-a600-4b88-b4d6-ca0e777a4ff1 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:49.867360 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba41a36-d535-4f6c-b304-7d002afacf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:49.867968 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba41a36-d535-4f6c-b304-7d002afacf97 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:49.868278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba41a36-d535-4f6c-b304-7d002afacf97 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:49.868548 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba41a36-d535-4f6c-b304-7d002afacf97 None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:49.869176 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba41a36-d535-4f6c-b304-7d002afacf97 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:49.869563 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba41a36-d535-4f6c-b304-7d002afacf97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:49.870937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba41a36-d535-4f6c-b304-7d002afacf97 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:49.875944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 642/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Sun Aug 7 19:16:49 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:49.882081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73ea37-5f8c-4ded-ad84-58eb46aa6b65 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:49.989596 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73ea37-5f8c-4ded-ad84-58eb46aa6b65 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:50.023713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdf239d-7202-4b8f-833a-9e259e6cf728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:50.024360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf239d-7202-4b8f-833a-9e259e6cf728 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:50.024631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf239d-7202-4b8f-833a-9e259e6cf728 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:50.024897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdf239d-7202-4b8f-833a-9e259e6cf728 None admin] PATH_INFO: `/v3/services` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:50.025522 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf239d-7202-4b8f-833a-9e259e6cf728 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:50.025843 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf239d-7202-4b8f-833a-9e259e6cf728 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:50.027189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdf239d-7202-4b8f-833a-9e259e6cf728 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:50.031679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 650/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Sun Aug 7 19:16:49 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:51.993871 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812e01b-9de1-4043-a554-f7f9bd0b1a17 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:51.994503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812e01b-9de1-4043-a554-f7f9bd0b1a17 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:51.994503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812e01b-9de1-4043-a554-f7f9bd0b1a17 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:51.995909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 643/1293] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:16:52.001856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad6e22-8283-4879-9a72-af9ba4870a31 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:52.002353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad6e22-8283-4879-9a72-af9ba4870a31 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:52.002530 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad6e22-8283-4879-9a72-af9ba4870a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:52.030441 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-aaad6e22-8283-4879-9a72-af9ba4870a31 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:16:52.031655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-aaad6e22-8283-4879-9a72-af9ba4870a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:16:52.064958 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:52.108665 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 651/1294] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:16:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:16:52.131256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812e01b-9de1-4043-a554-f7f9bd0b1a17 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:52.165646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:52.241600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812e01b-9de1-4043-a554-f7f9bd0b1a17 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:52.277226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3b2d7-b33c-4ac1-a45f-e5b55fe50bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:52.277709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3b2d7-b33c-4ac1-a45f-e5b55fe50bc4 None admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:52.277981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3b2d7-b33c-4ac1-a45f-e5b55fe50bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:52.278279 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3b2d7-b33c-4ac1-a45f-e5b55fe50bc4 None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:52.278897 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3b2d7-b33c-4ac1-a45f-e5b55fe50bc4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:52.280335 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b3b2d7-b33c-4ac1-a45f-e5b55fe50bc4 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:52.296388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 644/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Sun Aug 7 19:16:52 2022] POST /identity/v3/services => generated 246 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:16:54.270806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45340b42-b615-4061-a070-6c420d3b2d86 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:54.271360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45340b42-b615-4061-a070-6c420d3b2d86 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:54.271360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45340b42-b615-4061-a070-6c420d3b2d86 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:54.272674 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 652/1296] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:16:54.277952 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402f5ca-c2d6-46c1-843a-d6f4cb44ba3c None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:54.278413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402f5ca-c2d6-46c1-843a-d6f4cb44ba3c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:54.278526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402f5ca-c2d6-46c1-843a-d6f4cb44ba3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:54.296985 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-9402f5ca-c2d6-46c1-843a-d6f4cb44ba3c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:16:54.298202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9402f5ca-c2d6-46c1-843a-d6f4cb44ba3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:16:54.378737 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 645/1297] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:16:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:16:54.401675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45340b42-b615-4061-a070-6c420d3b2d86 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:54.506905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45340b42-b615-4061-a070-6c420d3b2d86 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:54.543053 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e655761-96b9-4945-b79a-2cb1d3e86ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:54.543600 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e655761-96b9-4945-b79a-2cb1d3e86ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:54.543897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e655761-96b9-4945-b79a-2cb1d3e86ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:54.544196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e655761-96b9-4945-b79a-2cb1d3e86ce8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:54.544766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e655761-96b9-4945-b79a-2cb1d3e86ce8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:54.546148 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e655761-96b9-4945-b79a-2cb1d3e86ce8 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:54.555117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-1e655761-96b9-4945-b79a-2cb1d3e86ce8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 19:16:54.555117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:54.555117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:16:54.555117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:16:54.555117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:16:54.555117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:16:54.555117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:16:54.555117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:16:54.555117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 19:16:54.555117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:54.555117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:16:54.555117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:54.555117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:54.555117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:54.555117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:54.555117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:54.555117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:54.556675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:54.556675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:54.556675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:54.556675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:54.556675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:54.556675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:54.556675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:16:54.556675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:16:54.556675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:16:54.556675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:16:54.556675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:54.556675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:54.556675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:54.556675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:54.556675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:54.556675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:54.557560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:54.557560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:54.557560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:54.557560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:54.557560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:54.557560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:54.557560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:54.557560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:54.557560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:54.557560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:54.557560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:16:54.557560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:16:54.557560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 19:16:54.557560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:54.557560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 653/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Aug 7 19:16:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:54.563467 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402f5ca-c2d6-46c1-843a-d6f4cb44ba3c None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:54.584678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:54.672385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402f5ca-c2d6-46c1-843a-d6f4cb44ba3c None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:54.709217 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be562f-f2d4-4428-aaab-a8458c076236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:54.710017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be562f-f2d4-4428-aaab-a8458c076236 None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:54.710300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be562f-f2d4-4428-aaab-a8458c076236 None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:54.710604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0be562f-f2d4-4428-aaab-a8458c076236 None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:54.711321 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be562f-f2d4-4428-aaab-a8458c076236 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:54.711710 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be562f-f2d4-4428-aaab-a8458c076236 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:54.713348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0be562f-f2d4-4428-aaab-a8458c076236 None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:54.719819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 646/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Sun Aug 7 19:16:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:54.728264 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e655761-96b9-4945-b79a-2cb1d3e86ce8 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:54.766521 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:54.851585 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e655761-96b9-4945-b79a-2cb1d3e86ce8 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:54.886617 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4fb59-7574-4729-a3ff-bb4f586176bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:54.887222 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4fb59-7574-4729-a3ff-bb4f586176bc None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:54.887501 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4fb59-7574-4729-a3ff-bb4f586176bc None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:54.887770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4fb59-7574-4729-a3ff-bb4f586176bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:54.888502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4fb59-7574-4729-a3ff-bb4f586176bc None admin] RBAC: Adding query filter params (interface=public, service_id=88f9898952484f1fa568b860fd2eaa6c, region_id=RegionOne) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:54.888799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4fb59-7574-4729-a3ff-bb4f586176bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:54.890197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4fb59-7574-4729-a3ff-bb4f586176bc None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:54.897806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 654/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Sun Aug 7 19:16:54 2022] GET /identity/v3/endpoints?service_id=88f9898952484f1fa568b860fd2eaa6c&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:54.904115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be562f-f2d4-4428-aaab-a8458c076236 None admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:54.995901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:55.020117 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0be562f-f2d4-4428-aaab-a8458c076236 None admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:55.058703 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98e863-04b0-4abb-9113-dc0b178043fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:55.059321 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98e863-04b0-4abb-9113-dc0b178043fe None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:55.059600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98e863-04b0-4abb-9113-dc0b178043fe None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:55.059865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98e863-04b0-4abb-9113-dc0b178043fe None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:55.060579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b98e863-04b0-4abb-9113-dc0b178043fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:55.061944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b98e863-04b0-4abb-9113-dc0b178043fe None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:55.069898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 647/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Sun Aug 7 19:16:54 2022] GET /identity/v3/services => generated 2191 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:16:57.057772 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa199d9-6751-40c8-9998-54d201286b37 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:57.058348 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa199d9-6751-40c8-9998-54d201286b37 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:57.058348 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa199d9-6751-40c8-9998-54d201286b37 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:57.059679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 655/1302] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:16:57.065109 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c40f60-c39b-4721-b0fd-70e768b05b99 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:57.065537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c40f60-c39b-4721-b0fd-70e768b05b99 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:57.065655 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c40f60-c39b-4721-b0fd-70e768b05b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:57.084462 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-e6c40f60-c39b-4721-b0fd-70e768b05b99 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:16:57.085669 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e6c40f60-c39b-4721-b0fd-70e768b05b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:16:57.162582 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 648/1303] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:16:57 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 19:16:57.186060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa199d9-6751-40c8-9998-54d201286b37 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:57.288015 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:57.295422 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa199d9-6751-40c8-9998-54d201286b37 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:57.330778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07753eca-fc15-41cf-be04-0a0c85f26390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:57.331414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07753eca-fc15-41cf-be04-0a0c85f26390 None admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:57.331690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07753eca-fc15-41cf-be04-0a0c85f26390 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:57.331963 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07753eca-fc15-41cf-be04-0a0c85f26390 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:57.332579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07753eca-fc15-41cf-be04-0a0c85f26390 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:57.333964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07753eca-fc15-41cf-be04-0a0c85f26390 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:57.343035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application [None req-07753eca-fc15-41cf-be04-0a0c85f26390 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 19:16:57.343035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:57.343035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 19:16:57.343035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 19:16:57.343035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 19:16:57.343035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 19:16:57.343035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 19:16:57.343035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:16:57.343035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 19:16:57.343035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:57.343035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 19:16:57.343035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:57.343035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 19:16:57.343035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 19:16:57.343035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 19:16:57.343035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 19:16:57.343035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 19:16:57.344221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 19:16:57.344221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 19:16:57.344221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 19:16:57.344221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 19:16:57.344221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 19:16:57.344221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 19:16:57.344221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 19:16:57.344221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 19:16:57.344221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 19:16:57.344221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 19:16:57.344221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 19:16:57.344221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 19:16:57.344221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 19:16:57.344221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 19:16:57.344221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 19:16:57.344221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 19:16:57.345108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 19:16:57.345108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application with Lock( Aug 07 19:16:57.345108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 19:16:57.345108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self._enter() Aug 07 19:16:57.345108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 19:16:57.345108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 19:16:57.345108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 19:16:57.345108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application return self.creator() Aug 07 19:16:57.345108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 19:16:57.345108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application created_value = creator( Aug 07 19:16:57.345108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 19:16:57.345108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 19:16:57.345108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 19:16:57.345108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: ERROR keystone.server.flask.application Aug 07 19:16:57.345108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 656/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Sun Aug 7 19:16:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 19:16:57.350464 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c40f60-c39b-4721-b0fd-70e768b05b99 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:57.427611 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:16:57.454465 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c40f60-c39b-4721-b0fd-70e768b05b99 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:57.490919 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33abd0-7500-406e-a9ee-febacad5ad4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:57.491536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33abd0-7500-406e-a9ee-febacad5ad4f None admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:57.491810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33abd0-7500-406e-a9ee-febacad5ad4f None admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:57.492107 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33abd0-7500-406e-a9ee-febacad5ad4f None admin] PATH_INFO: `/v3/services` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:57.492736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33abd0-7500-406e-a9ee-febacad5ad4f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:16:57.493034 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33abd0-7500-406e-a9ee-febacad5ad4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:57.494401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33abd0-7500-406e-a9ee-febacad5ad4f None admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:57.499774 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 649/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Sun Aug 7 19:16:57 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:16:57.505892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07753eca-fc15-41cf-be04-0a0c85f26390 None admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:16:57.612426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07753eca-fc15-41cf-be04-0a0c85f26390 None admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:16:57.647622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83566840-7f2f-4774-b367-a0f9b368d780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:16:57.648257 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83566840-7f2f-4774-b367-a0f9b368d780 None admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:16:57.648530 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83566840-7f2f-4774-b367-a0f9b368d780 None admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:16:57.648792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83566840-7f2f-4774-b367-a0f9b368d780 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:16:57.649451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83566840-7f2f-4774-b367-a0f9b368d780 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:16:57.650820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83566840-7f2f-4774-b367-a0f9b368d780 None admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:16:57.673882 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 657/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:16:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:17:07.496961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=30 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:17:07.669245 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:01.487558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079a5c4-1030-4387-9e32-5a51c05bb085 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:01.489411 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079a5c4-1030-4387-9e32-5a51c05bb085 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:01.490182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0079a5c4-1030-4387-9e32-5a51c05bb085 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:01.493478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 650/1307] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:18:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:01.499025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf5930-c20f-4e30-ae2a-1816a2372e48 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:01.499802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf5930-c20f-4e30-ae2a-1816a2372e48 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:01.499802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf5930-c20f-4e30-ae2a-1816a2372e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:01.598603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-dadf5930-c20f-4e30-ae2a-1816a2372e48 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:01.600074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-dadf5930-c20f-4e30-ae2a-1816a2372e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:01.748589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 658/1308] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:18:01.826136 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5872a-3cfe-4d21-9a7d-4291e73bbc22 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:01.827174 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5872a-3cfe-4d21-9a7d-4291e73bbc22 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:01.827895 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f5872a-3cfe-4d21-9a7d-4291e73bbc22 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:01.831018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 651/1309] 10.222.0.17 () {54 vars in 965 bytes} [Sun Aug 7 19:18:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:01.837303 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123bf31-241a-4e50-abc5-8311171ff1d1 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:01.837864 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123bf31-241a-4e50-abc5-8311171ff1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:01.838273 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123bf31-241a-4e50-abc5-8311171ff1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:01.930645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-b123bf31-241a-4e50-abc5-8311171ff1d1 None None] MFA Rules not processed for user `08229e9400b14e8a9cc8e8c32dc48285`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:01.932580 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b123bf31-241a-4e50-abc5-8311171ff1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:01.957352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:02.054758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5872a-3cfe-4d21-9a7d-4291e73bbc22 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:18:02.057717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-85f5872a-3cfe-4d21-9a7d-4291e73bbc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:02.062714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 659/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Sun Aug 7 19:18:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:18:02.264991 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f5872a-3cfe-4d21-9a7d-4291e73bbc22 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:18:02.369158 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9af2d-001b-47de-912a-27ae513d0b8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:18:02.370569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9af2d-001b-47de-912a-27ae513d0b8b service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:02.371072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9af2d-001b-47de-912a-27ae513d0b8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:02.371534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9af2d-001b-47de-912a-27ae513d0b8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:02.373533 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-05a9af2d-001b-47de-912a-27ae513d0b8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:02.440248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:02.512053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a9af2d-001b-47de-912a-27ae513d0b8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:18:02.513833 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a9af2d-001b-47de-912a-27ae513d0b8b service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:18:02.538429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 652/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:18:05.397975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d2199-63d1-4cab-bf09-1dd42f765b54 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:05.398778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d2199-63d1-4cab-bf09-1dd42f765b54 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:05.398778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d2199-63d1-4cab-bf09-1dd42f765b54 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:05.400139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 660/1312] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:18:05.405875 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620bacb-49a6-4ef0-92cd-8095e983e215 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:05.406681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620bacb-49a6-4ef0-92cd-8095e983e215 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:05.406681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620bacb-49a6-4ef0-92cd-8095e983e215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:05.510917 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-d620bacb-49a6-4ef0-92cd-8095e983e215 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:05.514240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d620bacb-49a6-4ef0-92cd-8095e983e215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:05.664170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 653/1313] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:18:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:18:05.739173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd6a0d-2ba3-4a45-b127-44d0466edace None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:05.739295 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd6a0d-2ba3-4a45-b127-44d0466edace None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:05.739576 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd6a0d-2ba3-4a45-b127-44d0466edace None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:05.741267 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 661/1314] 10.222.0.17 () {54 vars in 965 bytes} [Sun Aug 7 19:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:05.748119 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56be17-cb30-497a-8128-4cce29628231 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:05.748305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56be17-cb30-497a-8128-4cce29628231 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:05.748583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56be17-cb30-497a-8128-4cce29628231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:05.763297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:05.848615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-de56be17-cb30-497a-8128-4cce29628231 None None] MFA Rules not processed for user `08229e9400b14e8a9cc8e8c32dc48285`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:05.850015 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-de56be17-cb30-497a-8128-4cce29628231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:05.959577 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 654/1315] 10.222.0.17 () {58 vars in 1062 bytes} [Sun Aug 7 19:18:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:18:05.977552 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd6a0d-2ba3-4a45-b127-44d0466edace None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:18:05.980071 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-18cd6a0d-2ba3-4a45-b127-44d0466edace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:06.146688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd6a0d-2ba3-4a45-b127-44d0466edace None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:18:06.235946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:06.239566 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75404116-cc74-4a7e-9ae1-f4f671db584b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:18:06.240364 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404116-cc74-4a7e-9ae1-f4f671db584b service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:06.240959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404116-cc74-4a7e-9ae1-f4f671db584b service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:06.241536 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75404116-cc74-4a7e-9ae1-f4f671db584b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:06.243451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-75404116-cc74-4a7e-9ae1-f4f671db584b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:06.369038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75404116-cc74-4a7e-9ae1-f4f671db584b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:18:06.370952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75404116-cc74-4a7e-9ae1-f4f671db584b service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:18:06.400197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 662/1316] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:18:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:18:08.989148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dc0a3-200b-438b-976c-6fb71956a969 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:08.989733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dc0a3-200b-438b-976c-6fb71956a969 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:08.989733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dc0a3-200b-438b-976c-6fb71956a969 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:08.991233 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 655/1317] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:08.997360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19ee46-8b9b-479a-aa4a-c9c01b993688 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:08.997942 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19ee46-8b9b-479a-aa4a-c9c01b993688 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:08.997942 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19ee46-8b9b-479a-aa4a-c9c01b993688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:09.076937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-2b19ee46-8b9b-479a-aa4a-c9c01b993688 None None] MFA Rules not processed for user `14ef987ec6c24ababb3cce74ff137945`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:09.078169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-2b19ee46-8b9b-479a-aa4a-c9c01b993688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:09.140679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:09.188816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 663/1318] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:18:08 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:18:09.228933 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dc0a3-200b-438b-976c-6fb71956a969 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:18:09.230554 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5f4dc0a3-200b-438b-976c-6fb71956a969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:09.343124 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dc0a3-200b-438b-976c-6fb71956a969 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:18:09.346025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:09.424226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e3d49-3c6c-4fed-b980-f5ec0fe002e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:18:09.424849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e3d49-3c6c-4fed-b980-f5ec0fe002e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:09.425129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e3d49-3c6c-4fed-b980-f5ec0fe002e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:09.425397 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e3d49-3c6c-4fed-b980-f5ec0fe002e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:09.427082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-243e3d49-3c6c-4fed-b980-f5ec0fe002e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:09.537896 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e3d49-3c6c-4fed-b980-f5ec0fe002e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:18:09.539343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e3d49-3c6c-4fed-b980-f5ec0fe002e6 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:18:09.558732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 656/1319] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:18:09 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:18:12.305500 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed0096-a5f4-4f6a-bc2d-cf17e08e3cdf None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:12.306041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed0096-a5f4-4f6a-bc2d-cf17e08e3cdf None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:12.306041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed0096-a5f4-4f6a-bc2d-cf17e08e3cdf None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:12.307655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 664/1320] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:12.313519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5407452-2861-4599-a168-54bc7219199e None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:12.314061 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5407452-2861-4599-a168-54bc7219199e None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:12.314061 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5407452-2861-4599-a168-54bc7219199e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:12.359717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:12.388253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-f5407452-2861-4599-a168-54bc7219199e None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:12.389505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f5407452-2861-4599-a168-54bc7219199e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:12.494113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 657/1321] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:18:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:18:12.533719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed0096-a5f4-4f6a-bc2d-cf17e08e3cdf None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:18:12.535301 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b6ed0096-a5f4-4f6a-bc2d-cf17e08e3cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:12.643860 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed0096-a5f4-4f6a-bc2d-cf17e08e3cdf None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:18:12.721698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd02fc-67d5-46f0-a1a6-177233427eb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:18:12.722358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd02fc-67d5-46f0-a1a6-177233427eb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:12.722615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd02fc-67d5-46f0-a1a6-177233427eb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:12.722961 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd02fc-67d5-46f0-a1a6-177233427eb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:12.724610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b6cd02fc-67d5-46f0-a1a6-177233427eb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:12.732909 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:12.847186 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd02fc-67d5-46f0-a1a6-177233427eb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:18:12.848743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cd02fc-67d5-46f0-a1a6-177233427eb5 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:18:12.868619 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 665/1322] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:18:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:18:15.658345 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7eacb5-78c0-47e6-8b7b-d472ad43978c None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:15.659003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7eacb5-78c0-47e6-8b7b-d472ad43978c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:15.659003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7eacb5-78c0-47e6-8b7b-d472ad43978c None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:15.660333 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 658/1323] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:15.666109 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00782ae-be95-46d0-9b3b-8a0bcf3af6e5 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:15.666673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00782ae-be95-46d0-9b3b-8a0bcf3af6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:15.666673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00782ae-be95-46d0-9b3b-8a0bcf3af6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:15.743889 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-a00782ae-be95-46d0-9b3b-8a0bcf3af6e5 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:15.745276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a00782ae-be95-46d0-9b3b-8a0bcf3af6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:15.812113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:15.865124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 666/1324] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:18:15.902943 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7eacb5-78c0-47e6-8b7b-d472ad43978c None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:18:15.904877 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-7e7eacb5-78c0-47e6-8b7b-d472ad43978c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:16.052969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7eacb5-78c0-47e6-8b7b-d472ad43978c None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:18:16.068313 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:16.140627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed3464-20f5-43af-a089-a8ee962e5504 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:18:16.141412 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed3464-20f5-43af-a089-a8ee962e5504 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:16.141676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed3464-20f5-43af-a089-a8ee962e5504 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:16.142007 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed3464-20f5-43af-a089-a8ee962e5504 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:16.143916 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-26ed3464-20f5-43af-a089-a8ee962e5504 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:16.278021 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed3464-20f5-43af-a089-a8ee962e5504 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:18:16.279367 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed3464-20f5-43af-a089-a8ee962e5504 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:18:16.298922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 659/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:18:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:18:18.540273 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c6854-b5cd-4073-aeed-191218a15c71 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:18.540894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c6854-b5cd-4073-aeed-191218a15c71 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:18.540894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c6854-b5cd-4073-aeed-191218a15c71 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:18.542314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 667/1326] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:18.548465 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fa418-09b5-447d-9340-fa6be9d5597f None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:18.549013 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fa418-09b5-447d-9340-fa6be9d5597f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:18.549013 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fa418-09b5-447d-9340-fa6be9d5597f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:18.603273 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:18.624260 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-b93fa418-09b5-447d-9340-fa6be9d5597f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:18.625791 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b93fa418-09b5-447d-9340-fa6be9d5597f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:18.740445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 660/1327] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:18:18.777132 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c6854-b5cd-4073-aeed-191218a15c71 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:18:18.778711 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c42c6854-b5cd-4073-aeed-191218a15c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:18.894298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c6854-b5cd-4073-aeed-191218a15c71 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:18:18.976503 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff069918-2642-412a-8320-c3005a60d0a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:18:18.977134 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff069918-2642-412a-8320-c3005a60d0a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:18.977415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff069918-2642-412a-8320-c3005a60d0a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:18.977683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff069918-2642-412a-8320-c3005a60d0a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:18.979321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-ff069918-2642-412a-8320-c3005a60d0a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:18.998149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:19.105876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff069918-2642-412a-8320-c3005a60d0a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:18:19.107387 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff069918-2642-412a-8320-c3005a60d0a8 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:18:19.127835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 668/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:18:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:18:21.221768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69e7fd-5245-41c3-877d-28977b4f344e None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:21.222349 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69e7fd-5245-41c3-877d-28977b4f344e None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:21.222349 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69e7fd-5245-41c3-877d-28977b4f344e None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:21.223731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 661/1329] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:21.229568 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935f7d2-e736-457a-b72d-ea09776aeac1 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:21.230473 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935f7d2-e736-457a-b72d-ea09776aeac1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:21.230745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935f7d2-e736-457a-b72d-ea09776aeac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:21.307237 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-2935f7d2-e736-457a-b72d-ea09776aeac1 None None] MFA Rules not processed for user `14ef987ec6c24ababb3cce74ff137945`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:21.308507 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-2935f7d2-e736-457a-b72d-ea09776aeac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:21.377394 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:21.417910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 669/1330] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:18:21 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:18:21.459121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69e7fd-5245-41c3-877d-28977b4f344e None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:18:21.461931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ea69e7fd-5245-41c3-877d-28977b4f344e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:21.578897 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69e7fd-5245-41c3-877d-28977b4f344e None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:18:21.599947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:21.676188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9710be-ac61-4f80-b6b6-92ed0dad18ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:18:21.676786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9710be-ac61-4f80-b6b6-92ed0dad18ce service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:21.677059 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9710be-ac61-4f80-b6b6-92ed0dad18ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:21.677325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9710be-ac61-4f80-b6b6-92ed0dad18ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:21.678963 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5a9710be-ac61-4f80-b6b6-92ed0dad18ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:21.803205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9710be-ac61-4f80-b6b6-92ed0dad18ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:18:21.804688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9710be-ac61-4f80-b6b6-92ed0dad18ce service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:18:21.824859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 662/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:18:21 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:18:24.126595 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcf5e4-bdb4-4394-857c-ac37001c5f0b None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:24.127190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcf5e4-bdb4-4394-857c-ac37001c5f0b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:24.127190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcf5e4-bdb4-4394-857c-ac37001c5f0b None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:24.128730 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 670/1332] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:24.134301 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a2e20-0148-4e1e-916d-f51f45b2089f None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:24.134840 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a2e20-0148-4e1e-916d-f51f45b2089f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:24.134840 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a2e20-0148-4e1e-916d-f51f45b2089f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:24.203887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:24.208015 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-b02a2e20-0148-4e1e-916d-f51f45b2089f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:24.210007 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b02a2e20-0148-4e1e-916d-f51f45b2089f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:24.310997 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 663/1333] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:18:24.347121 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bcf5e4-bdb4-4394-857c-ac37001c5f0b None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:18:24.348961 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f5bcf5e4-bdb4-4394-857c-ac37001c5f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:24.457843 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bcf5e4-bdb4-4394-857c-ac37001c5f0b None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:18:24.530331 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2c5b3-4d7a-47f1-9b6d-3b5d37d20fdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:18:24.530956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2c5b3-4d7a-47f1-9b6d-3b5d37d20fdc service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:24.531228 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2c5b3-4d7a-47f1-9b6d-3b5d37d20fdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:24.531474 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2c5b3-4d7a-47f1-9b6d-3b5d37d20fdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:24.533189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b4e2c5b3-4d7a-47f1-9b6d-3b5d37d20fdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:24.566137 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:24.656722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2c5b3-4d7a-47f1-9b6d-3b5d37d20fdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:18:24.658195 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2c5b3-4d7a-47f1-9b6d-3b5d37d20fdc service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:18:24.677697 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 671/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:18:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:18:26.781257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a71ff-7761-4337-aa6f-0089f548d31d None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:26.781882 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a71ff-7761-4337-aa6f-0089f548d31d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:26.781882 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a71ff-7761-4337-aa6f-0089f548d31d None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:26.783166 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 664/1335] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:26.789139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45c4f5-337a-4966-8b49-00ed539e2d1c None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:26.789638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45c4f5-337a-4966-8b49-00ed539e2d1c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:26.789638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd45c4f5-337a-4966-8b49-00ed539e2d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:26.863590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-cd45c4f5-337a-4966-8b49-00ed539e2d1c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:26.864835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-cd45c4f5-337a-4966-8b49-00ed539e2d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:26.951507 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:26.974579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 672/1336] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:18:27.015798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a71ff-7761-4337-aa6f-0089f548d31d None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:18:27.017410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e00a71ff-7761-4337-aa6f-0089f548d31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:27.128078 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a71ff-7761-4337-aa6f-0089f548d31d None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:18:27.158389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:27.205473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d901fd9-d738-4c74-8443-05ff96cd8773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:18:27.206089 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d901fd9-d738-4c74-8443-05ff96cd8773 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:27.206366 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d901fd9-d738-4c74-8443-05ff96cd8773 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:27.206636 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d901fd9-d738-4c74-8443-05ff96cd8773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:27.208404 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6d901fd9-d738-4c74-8443-05ff96cd8773 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:27.319855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d901fd9-d738-4c74-8443-05ff96cd8773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:18:27.321346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d901fd9-d738-4c74-8443-05ff96cd8773 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:18:27.342630 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 665/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:18:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:18:29.899946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ad267-776c-450a-bf69-1535a14366dc None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:29.900609 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ad267-776c-450a-bf69-1535a14366dc None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:29.900609 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ad267-776c-450a-bf69-1535a14366dc None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:29.901948 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 673/1338] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:29.907535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc84359-49a8-4a5c-b835-b02aff269da4 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:29.908003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc84359-49a8-4a5c-b835-b02aff269da4 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:29.908120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc84359-49a8-4a5c-b835-b02aff269da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:30.015447 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-1dc84359-49a8-4a5c-b835-b02aff269da4 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:30.016716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1dc84359-49a8-4a5c-b835-b02aff269da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:30.025862 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:30.132682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 666/1339] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:18:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:18:30.169579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ad267-776c-450a-bf69-1535a14366dc None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:18:30.171227 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0c8ad267-776c-450a-bf69-1535a14366dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:30.303718 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ad267-776c-450a-bf69-1535a14366dc None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:18:30.377865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f47e2f-8e76-4b44-9fe6-3930e48adbec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:18:30.378471 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f47e2f-8e76-4b44-9fe6-3930e48adbec service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:30.378744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f47e2f-8e76-4b44-9fe6-3930e48adbec service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:30.379060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f47e2f-8e76-4b44-9fe6-3930e48adbec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:30.380855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a5f47e2f-8e76-4b44-9fe6-3930e48adbec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:30.426129 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:30.508511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f47e2f-8e76-4b44-9fe6-3930e48adbec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:18:30.509964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f47e2f-8e76-4b44-9fe6-3930e48adbec service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:18:30.528642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 674/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:18:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:18:34.529621 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4165d-5cd0-4868-b11a-64f6518941c6 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:34.530206 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4165d-5cd0-4868-b11a-64f6518941c6 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:34.530315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4165d-5cd0-4868-b11a-64f6518941c6 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:34.531624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 667/1341] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:34.537233 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99979332-6f9d-46d1-83fe-5b5cf90b9729 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:34.537704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99979332-6f9d-46d1-83fe-5b5cf90b9729 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:34.537903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99979332-6f9d-46d1-83fe-5b5cf90b9729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:34.606022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-99979332-6f9d-46d1-83fe-5b5cf90b9729 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:34.607284 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-99979332-6f9d-46d1-83fe-5b5cf90b9729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:34.697118 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:34.710850 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 675/1342] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:18:34.751116 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4165d-5cd0-4868-b11a-64f6518941c6 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:18:34.752827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-11f4165d-5cd0-4868-b11a-64f6518941c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:34.860256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4165d-5cd0-4868-b11a-64f6518941c6 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:18:34.900049 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:34.937978 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d857f52-30df-4442-86bd-ac147beb4ed7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:18:34.938620 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d857f52-30df-4442-86bd-ac147beb4ed7 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:34.938901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d857f52-30df-4442-86bd-ac147beb4ed7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:34.939171 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d857f52-30df-4442-86bd-ac147beb4ed7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:34.940835 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6d857f52-30df-4442-86bd-ac147beb4ed7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:35.053119 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d857f52-30df-4442-86bd-ac147beb4ed7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:18:35.054575 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d857f52-30df-4442-86bd-ac147beb4ed7 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:18:35.073975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 668/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:18:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:18:37.658649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3303-0832-4ad8-893a-889a1960c451 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:37.659289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3303-0832-4ad8-893a-889a1960c451 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:37.659289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3303-0832-4ad8-893a-889a1960c451 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:37.660947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 676/1344] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:18:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:37.667158 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4e71c-c4e6-4629-ab22-8a6d0bcd7fc7 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:37.667638 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4e71c-c4e6-4629-ab22-8a6d0bcd7fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:37.667786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd4e71c-c4e6-4629-ab22-8a6d0bcd7fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:37.748885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-3bd4e71c-c4e6-4629-ab22-8a6d0bcd7fc7 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:37.750154 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-3bd4e71c-c4e6-4629-ab22-8a6d0bcd7fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:37.772264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:37.871724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 669/1345] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:18:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:18:37.912807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e3303-0832-4ad8-893a-889a1960c451 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:18:37.915631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-720e3303-0832-4ad8-893a-889a1960c451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:38.064689 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e3303-0832-4ad8-893a-889a1960c451 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:18:38.146460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229734f-41d2-4063-a3da-6e83dea5688f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:18:38.147081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229734f-41d2-4063-a3da-6e83dea5688f service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:38.147359 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229734f-41d2-4063-a3da-6e83dea5688f service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:38.147627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229734f-41d2-4063-a3da-6e83dea5688f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:38.149303 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e229734f-41d2-4063-a3da-6e83dea5688f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:38.203849 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:38.293960 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e229734f-41d2-4063-a3da-6e83dea5688f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:18:38.295597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e229734f-41d2-4063-a3da-6e83dea5688f service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:18:38.320356 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 677/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:18:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:18:42.876109 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf5fa2-6aff-42f7-b07d-b990c63a53b7 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:42.876645 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf5fa2-6aff-42f7-b07d-b990c63a53b7 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:42.876645 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf5fa2-6aff-42f7-b07d-b990c63a53b7 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:42.878028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 670/1347] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:42.883867 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941d500-34ad-4d74-91f7-a7993b6cc676 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:42.884393 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941d500-34ad-4d74-91f7-a7993b6cc676 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:42.884488 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941d500-34ad-4d74-91f7-a7993b6cc676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:42.956720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-8941d500-34ad-4d74-91f7-a7993b6cc676 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:42.958035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8941d500-34ad-4d74-91f7-a7993b6cc676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:43.080522 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:43.085805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 678/1348] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:18:43.125305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf5fa2-6aff-42f7-b07d-b990c63a53b7 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:18:43.126906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-cacf5fa2-6aff-42f7-b07d-b990c63a53b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:43.237403 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf5fa2-6aff-42f7-b07d-b990c63a53b7 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:18:43.287764 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:43.313657 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af333959-ea4c-4cc8-a1e7-fbab024c5410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:18:43.314272 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af333959-ea4c-4cc8-a1e7-fbab024c5410 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:43.314543 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af333959-ea4c-4cc8-a1e7-fbab024c5410 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:43.314836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af333959-ea4c-4cc8-a1e7-fbab024c5410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:43.316609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-af333959-ea4c-4cc8-a1e7-fbab024c5410 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:43.435421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af333959-ea4c-4cc8-a1e7-fbab024c5410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:18:43.436828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af333959-ea4c-4cc8-a1e7-fbab024c5410 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:18:43.455397 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 671/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:18:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:18:48.529361 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57656f0-8b0f-43e3-aec5-2349157a6656 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:48.530351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57656f0-8b0f-43e3-aec5-2349157a6656 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:48.530845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57656f0-8b0f-43e3-aec5-2349157a6656 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:48.532702 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 679/1350] 10.222.0.17 () {54 vars in 921 bytes} [Sun Aug 7 19:18:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:48.538744 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fe42c-9076-4e02-973c-a930b3cb09ca None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:48.539264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fe42c-9076-4e02-973c-a930b3cb09ca None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:48.539527 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fe42c-9076-4e02-973c-a930b3cb09ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:48.672120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-495fe42c-9076-4e02-973c-a930b3cb09ca None None] MFA Rules not processed for user `143c30b4201645279564e72ab3288adc`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:48.673545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-495fe42c-9076-4e02-973c-a930b3cb09ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:48.707415 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:48.850323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 672/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Aug 7 19:18:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:18:52.895109 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961b704-ec5a-490a-a4bf-4192cf21be38 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:52.895706 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961b704-ec5a-490a-a4bf-4192cf21be38 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:52.895706 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961b704-ec5a-490a-a4bf-4192cf21be38 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:52.897318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 680/1352] 10.222.0.17 () {54 vars in 921 bytes} [Sun Aug 7 19:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:52.902491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12005a-639e-424a-9e83-b5c8720e5cb7 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:52.902980 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12005a-639e-424a-9e83-b5c8720e5cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:52.903249 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12005a-639e-424a-9e83-b5c8720e5cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:53.000222 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-1e12005a-639e-424a-9e83-b5c8720e5cb7 None None] MFA Rules not processed for user `143c30b4201645279564e72ab3288adc`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:53.001625 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1e12005a-639e-424a-9e83-b5c8720e5cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:53.080708 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:53.122115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 673/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Aug 7 19:18:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:18:54.627286 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57787f-424c-4869-8429-d3d205c391c0 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:54.627840 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57787f-424c-4869-8429-d3d205c391c0 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:54.627840 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57787f-424c-4869-8429-d3d205c391c0 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:54.629220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 681/1354] 10.222.0.17 () {54 vars in 921 bytes} [Sun Aug 7 19:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:54.634368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b077ea8-0f5f-474e-9edf-173d11ba8b78 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:54.634838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b077ea8-0f5f-474e-9edf-173d11ba8b78 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:54.634927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b077ea8-0f5f-474e-9edf-173d11ba8b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:54.712756 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-7b077ea8-0f5f-474e-9edf-173d11ba8b78 None None] MFA Rules not processed for user `143c30b4201645279564e72ab3288adc`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:54.713985 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-7b077ea8-0f5f-474e-9edf-173d11ba8b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:54.825194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 674/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Aug 7 19:18:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:18:58.186028 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f96cf-072c-46fa-8fca-f0befd5d4454 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:58.186613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f96cf-072c-46fa-8fca-f0befd5d4454 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:58.186613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f96cf-072c-46fa-8fca-f0befd5d4454 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:58.187982 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 682/1356] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:58.195297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa2ac0-b2f6-4921-9aa1-216ef8e1233a None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:58.195842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa2ac0-b2f6-4921-9aa1-216ef8e1233a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:58.195842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa2ac0-b2f6-4921-9aa1-216ef8e1233a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:58.251215 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:58.300566 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-22aa2ac0-b2f6-4921-9aa1-216ef8e1233a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:58.303756 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-22aa2ac0-b2f6-4921-9aa1-216ef8e1233a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:58.412051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 675/1357] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:18:58.466168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba3ec7-2780-4591-8384-522be2b0c868 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:58.467059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba3ec7-2780-4591-8384-522be2b0c868 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:58.467698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba3ec7-2780-4591-8384-522be2b0c868 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:58.470254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 683/1358] 10.222.0.17 () {54 vars in 962 bytes} [Sun Aug 7 19:18:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:18:58.477999 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e974e50-a5dc-4a90-b682-1d96fdf4c0eb None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:58.478289 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e974e50-a5dc-4a90-b682-1d96fdf4c0eb None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:58.478559 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e974e50-a5dc-4a90-b682-1d96fdf4c0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:58.572507 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-1e974e50-a5dc-4a90-b682-1d96fdf4c0eb None None] MFA Rules not processed for user `a4946aec299f45bd8402cff3159d04d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:18:58.573735 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1e974e50-a5dc-4a90-b682-1d96fdf4c0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:58.627219 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:58.713576 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 676/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Sun Aug 7 19:18:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:18:58.723126 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba3ec7-2780-4591-8384-522be2b0c868 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:18:58.724987 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c6ba3ec7-2780-4591-8384-522be2b0c868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:58.876825 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba3ec7-2780-4591-8384-522be2b0c868 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:18:58.954388 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:18:58.997742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0a033-9f6a-4f40-8f1b-9375741751f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:18:58.998274 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0a033-9f6a-4f40-8f1b-9375741751f0 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:18:58.998553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0a033-9f6a-4f40-8f1b-9375741751f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:18:58.998821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0a033-9f6a-4f40-8f1b-9375741751f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:18:59.000619 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3ac0a033-9f6a-4f40-8f1b-9375741751f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:18:59.102542 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0a033-9f6a-4f40-8f1b-9375741751f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:18:59.104261 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0a033-9f6a-4f40-8f1b-9375741751f0 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:18:59.125765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 684/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:19:01.338019 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254345b-c8d4-4d91-b0af-ec75586508df None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:01.338612 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254345b-c8d4-4d91-b0af-ec75586508df None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:01.338612 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254345b-c8d4-4d91-b0af-ec75586508df None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:01.340139 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 677/1361] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:01.345730 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece6f9f-7851-4a28-9053-d900b99a6241 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:01.346197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece6f9f-7851-4a28-9053-d900b99a6241 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:01.346197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece6f9f-7851-4a28-9053-d900b99a6241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:01.419414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-9ece6f9f-7851-4a28-9053-d900b99a6241 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:01.420660 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9ece6f9f-7851-4a28-9053-d900b99a6241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:01.462248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:01.534064 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 685/1362] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:19:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:19:01.579480 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254345b-c8d4-4d91-b0af-ec75586508df None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:01.581049 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4254345b-c8d4-4d91-b0af-ec75586508df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:01.809698 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254345b-c8d4-4d91-b0af-ec75586508df None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:01.832054 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:01.964129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caabf94-0b63-455f-bc14-8dc66300af1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:01.964764 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caabf94-0b63-455f-bc14-8dc66300af1a service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:01.965037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caabf94-0b63-455f-bc14-8dc66300af1a service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:01.965304 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caabf94-0b63-455f-bc14-8dc66300af1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:01.966961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6caabf94-0b63-455f-bc14-8dc66300af1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:02.073317 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caabf94-0b63-455f-bc14-8dc66300af1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:02.075095 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caabf94-0b63-455f-bc14-8dc66300af1a service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:02.102838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 678/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:19:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:19:04.446149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac963cd-890d-4d5d-9e81-87ec3b7aa256 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:04.446715 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac963cd-890d-4d5d-9e81-87ec3b7aa256 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:04.446715 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac963cd-890d-4d5d-9e81-87ec3b7aa256 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:04.448264 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 686/1364] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:04.453779 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d256e-8060-440e-81d8-b92ad5fad7d5 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:04.454272 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d256e-8060-440e-81d8-b92ad5fad7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:04.454368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598d256e-8060-440e-81d8-b92ad5fad7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:04.472668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:04.561007 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-598d256e-8060-440e-81d8-b92ad5fad7d5 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:04.562436 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-598d256e-8060-440e-81d8-b92ad5fad7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:04.817672 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 679/1365] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:19:04.865441 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ceb18-6dc3-4767-89ca-fd5cbccc3173 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:04.866377 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ceb18-6dc3-4767-89ca-fd5cbccc3173 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:04.866807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ceb18-6dc3-4767-89ca-fd5cbccc3173 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:04.868575 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 687/1366] 10.222.0.17 () {54 vars in 962 bytes} [Sun Aug 7 19:19:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:04.879250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8ae76-bea2-4f06-a418-96bce1add858 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:04.879709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8ae76-bea2-4f06-a418-96bce1add858 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:04.880165 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8ae76-bea2-4f06-a418-96bce1add858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:04.959582 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-90d8ae76-bea2-4f06-a418-96bce1add858 None None] MFA Rules not processed for user `a4946aec299f45bd8402cff3159d04d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:04.960850 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-90d8ae76-bea2-4f06-a418-96bce1add858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:04.999278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:05.110488 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 680/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Sun Aug 7 19:19:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:19:05.119097 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ceb18-6dc3-4767-89ca-fd5cbccc3173 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:05.120803 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-261ceb18-6dc3-4767-89ca-fd5cbccc3173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:05.280121 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ceb18-6dc3-4767-89ca-fd5cbccc3173 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:05.301481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:05.401481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca95dc-e7f3-4423-a145-45b5d128f5d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:05.402181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca95dc-e7f3-4423-a145-45b5d128f5d0 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:05.402580 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca95dc-e7f3-4423-a145-45b5d128f5d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:05.402937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca95dc-e7f3-4423-a145-45b5d128f5d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:05.404661 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e7ca95dc-e7f3-4423-a145-45b5d128f5d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:05.500855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca95dc-e7f3-4423-a145-45b5d128f5d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:05.502541 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca95dc-e7f3-4423-a145-45b5d128f5d0 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:05.523815 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 688/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:19:06.413016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ff3b2-9607-42e3-b144-71ab129b9e48 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:06.413802 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ff3b2-9607-42e3-b144-71ab129b9e48 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:06.413802 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ff3b2-9607-42e3-b144-71ab129b9e48 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:06.415049 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 681/1369] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:19:06.420341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346518b-7e34-477c-9263-5db85804dd55 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:06.420723 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346518b-7e34-477c-9263-5db85804dd55 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:06.420881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346518b-7e34-477c-9263-5db85804dd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:06.453565 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:06.491178 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-4346518b-7e34-477c-9263-5db85804dd55 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:06.492440 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4346518b-7e34-477c-9263-5db85804dd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:06.598138 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 689/1370] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:19:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:19:06.954595 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9ccd3-8bed-43f9-b7c7-57ffd6002bfb None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:06.954595 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9ccd3-8bed-43f9-b7c7-57ffd6002bfb None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:06.954595 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb9ccd3-8bed-43f9-b7c7-57ffd6002bfb None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:06.957928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 682/1371] 10.222.0.17 () {54 vars in 964 bytes} [Sun Aug 7 19:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:06.960510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc953896-ed53-4afc-9608-361c88509f4c None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:06.960769 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc953896-ed53-4afc-9608-361c88509f4c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:06.961075 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc953896-ed53-4afc-9608-361c88509f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:07.047746 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-fc953896-ed53-4afc-9608-361c88509f4c None None] MFA Rules not processed for user `f83e5f0b1d5e4f57bbf283d420f080ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:07.049009 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-fc953896-ed53-4afc-9608-361c88509f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:07.099559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:07.137458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 690/1372] 10.222.0.17 () {58 vars in 1061 bytes} [Sun Aug 7 19:19:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:19:07.145555 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9ccd3-8bed-43f9-b7c7-57ffd6002bfb None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:07.147095 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6cb9ccd3-8bed-43f9-b7c7-57ffd6002bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:07.273379 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb9ccd3-8bed-43f9-b7c7-57ffd6002bfb None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:07.324178 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:07.363834 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59a151-6cff-47a0-b25a-fbaa5d073bd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:07.364712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59a151-6cff-47a0-b25a-fbaa5d073bd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:07.365058 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59a151-6cff-47a0-b25a-fbaa5d073bd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:07.365231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59a151-6cff-47a0-b25a-fbaa5d073bd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:07.366882 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5f59a151-6cff-47a0-b25a-fbaa5d073bd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:07.495538 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59a151-6cff-47a0-b25a-fbaa5d073bd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:07.497036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59a151-6cff-47a0-b25a-fbaa5d073bd1 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:07.520402 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 683/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:19:07.663441 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb256e9d-aac6-49f6-b8c8-3b50fb5a0497 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:07.663940 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb256e9d-aac6-49f6-b8c8-3b50fb5a0497 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:07.665212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb256e9d-aac6-49f6-b8c8-3b50fb5a0497 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:07.676669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 691/1374] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:19:07 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:07.677262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1a72e-e2e2-49b9-9d44-ae33d9980f19 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:07.677262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1a72e-e2e2-49b9-9d44-ae33d9980f19 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:07.677262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1a72e-e2e2-49b9-9d44-ae33d9980f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:07.764571 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-65e1a72e-e2e2-49b9-9d44-ae33d9980f19 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:07.767352 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-65e1a72e-e2e2-49b9-9d44-ae33d9980f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:07.802133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:07.887230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 684/1375] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:19:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:19:07.932689 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb256e9d-aac6-49f6-b8c8-3b50fb5a0497 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:07.934253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-fb256e9d-aac6-49f6-b8c8-3b50fb5a0497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:08.096480 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb256e9d-aac6-49f6-b8c8-3b50fb5a0497 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:08.150414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:08.213826 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b909c-cbd3-415f-bf1d-21e03df3d4a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:08.214557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b909c-cbd3-415f-bf1d-21e03df3d4a8 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:08.214935 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b909c-cbd3-415f-bf1d-21e03df3d4a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:08.215315 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b909c-cbd3-415f-bf1d-21e03df3d4a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:08.217028 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9f9b909c-cbd3-415f-bf1d-21e03df3d4a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:08.316781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b909c-cbd3-415f-bf1d-21e03df3d4a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:08.318432 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b909c-cbd3-415f-bf1d-21e03df3d4a8 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:08.337868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 692/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:19:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:19:10.369873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2ba0c-55fa-4a8d-9eef-58a927282983 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:10.370421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2ba0c-55fa-4a8d-9eef-58a927282983 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:10.370421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2ba0c-55fa-4a8d-9eef-58a927282983 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:10.371690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 685/1377] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:10.377722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60e72c-fda0-465e-b533-c93b55d53cbe None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:10.378268 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60e72c-fda0-465e-b533-c93b55d53cbe None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:10.378268 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60e72c-fda0-465e-b533-c93b55d53cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:10.457675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-9f60e72c-fda0-465e-b533-c93b55d53cbe None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:10.458911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9f60e72c-fda0-465e-b533-c93b55d53cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:10.467355 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:10.581130 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 693/1378] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:19:10.629688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2ba0c-55fa-4a8d-9eef-58a927282983 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:10.631630 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-aae2ba0c-55fa-4a8d-9eef-58a927282983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:10.845800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2ba0c-55fa-4a8d-9eef-58a927282983 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:10.850576 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:10.987068 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4825d91f-5aa8-4775-b296-529f8ec28aec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:10.987707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825d91f-5aa8-4775-b296-529f8ec28aec service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:10.987985 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825d91f-5aa8-4775-b296-529f8ec28aec service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:10.988434 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4825d91f-5aa8-4775-b296-529f8ec28aec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:10.990277 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4825d91f-5aa8-4775-b296-529f8ec28aec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:11.126074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825d91f-5aa8-4775-b296-529f8ec28aec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:11.127154 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4825d91f-5aa8-4775-b296-529f8ec28aec service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:11.172516 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 686/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:19:13.319440 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f671e-e0a0-40c7-ad37-9b65e45bf4d7 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:13.320315 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f671e-e0a0-40c7-ad37-9b65e45bf4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:13.320315 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f671e-e0a0-40c7-ad37-9b65e45bf4d7 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:13.321395 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 694/1380] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:13.327549 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f9676-8492-4fb1-a720-c8aa719b7db5 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:13.328337 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f9676-8492-4fb1-a720-c8aa719b7db5 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:13.328337 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f9676-8492-4fb1-a720-c8aa719b7db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:13.349325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:13.411439 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-351f9676-8492-4fb1-a720-c8aa719b7db5 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:13.412698 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-351f9676-8492-4fb1-a720-c8aa719b7db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:13.520181 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 687/1381] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:19:13.563923 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f671e-e0a0-40c7-ad37-9b65e45bf4d7 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:13.565555 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8a3f671e-e0a0-40c7-ad37-9b65e45bf4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:13.934927 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:13.947562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f671e-e0a0-40c7-ad37-9b65e45bf4d7 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:14.108219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84b959-fd16-41e8-aa53-b5ddf70c38ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:14.109239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84b959-fd16-41e8-aa53-b5ddf70c38ad service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:14.109774 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84b959-fd16-41e8-aa53-b5ddf70c38ad service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:14.110190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84b959-fd16-41e8-aa53-b5ddf70c38ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:14.112563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3f84b959-fd16-41e8-aa53-b5ddf70c38ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:14.240378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84b959-fd16-41e8-aa53-b5ddf70c38ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:14.243579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f84b959-fd16-41e8-aa53-b5ddf70c38ad service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:14.259811 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:14.284517 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 695/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 721 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:19:16.578493 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189882b8-c6ad-4fc9-a6ec-4c1eb7cc2405 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:16.579857 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189882b8-c6ad-4fc9-a6ec-4c1eb7cc2405 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:16.579857 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189882b8-c6ad-4fc9-a6ec-4c1eb7cc2405 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:16.580500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 688/1383] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:19:16.588440 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b10e75-e8da-4778-8dc1-f6a82bad224d None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:16.589605 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b10e75-e8da-4778-8dc1-f6a82bad224d None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:16.589605 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b10e75-e8da-4778-8dc1-f6a82bad224d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:16.690521 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-f0b10e75-e8da-4778-8dc1-f6a82bad224d None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:16.691838 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f0b10e75-e8da-4778-8dc1-f6a82bad224d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:16.816696 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 696/1384] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:19:16.903541 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189882b8-c6ad-4fc9-a6ec-4c1eb7cc2405 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:16.905388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-189882b8-c6ad-4fc9-a6ec-4c1eb7cc2405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:17.033359 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:17.125436 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189882b8-c6ad-4fc9-a6ec-4c1eb7cc2405 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:17.295386 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4a54b-969b-4ff6-8434-310893e2b096 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:17.296317 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4a54b-969b-4ff6-8434-310893e2b096 service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:17.296681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4a54b-969b-4ff6-8434-310893e2b096 service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:17.297053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4a54b-969b-4ff6-8434-310893e2b096 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:17.299368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-bbb4a54b-969b-4ff6-8434-310893e2b096 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:17.369230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:17.421675 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4a54b-969b-4ff6-8434-310893e2b096 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:17.423868 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4a54b-969b-4ff6-8434-310893e2b096 service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:17.449872 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 689/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:19:19.570165 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47fb0d-afaa-48a5-9df0-0be7df4b4c15 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:19.570895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47fb0d-afaa-48a5-9df0-0be7df4b4c15 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:19.570895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47fb0d-afaa-48a5-9df0-0be7df4b4c15 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:19.572281 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 697/1386] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:19:19.579099 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8e494-dfc6-4799-9abd-b89d4fc6c686 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:19.579963 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8e494-dfc6-4799-9abd-b89d4fc6c686 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:19.579963 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8e494-dfc6-4799-9abd-b89d4fc6c686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:19.681218 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-03a8e494-dfc6-4799-9abd-b89d4fc6c686 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:19.683166 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-03a8e494-dfc6-4799-9abd-b89d4fc6c686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:19.811572 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 690/1387] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:19:19.859806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47fb0d-afaa-48a5-9df0-0be7df4b4c15 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:19.861470 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4c47fb0d-afaa-48a5-9df0-0be7df4b4c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:19.947695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:20.030954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47fb0d-afaa-48a5-9df0-0be7df4b4c15 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:20.155396 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1226544-4adb-4277-8638-f700a618c6d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:20.155954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1226544-4adb-4277-8638-f700a618c6d9 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:20.156256 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1226544-4adb-4277-8638-f700a618c6d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:20.156682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1226544-4adb-4277-8638-f700a618c6d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:20.158189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b1226544-4adb-4277-8638-f700a618c6d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:20.213448 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:20.273198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1226544-4adb-4277-8638-f700a618c6d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:20.274860 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1226544-4adb-4277-8638-f700a618c6d9 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:20.293925 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 698/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:19:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:19:22.345378 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9cb9d-1030-4730-9730-017d43d63fe9 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:22.346291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9cb9d-1030-4730-9730-017d43d63fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:22.346574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9cb9d-1030-4730-9730-017d43d63fe9 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:22.348060 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 691/1389] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:19:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:22.353874 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3af68-6e7c-42a2-82aa-4c513b80fd74 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:22.354436 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3af68-6e7c-42a2-82aa-4c513b80fd74 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:22.354436 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3af68-6e7c-42a2-82aa-4c513b80fd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:22.426014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-8ad3af68-6e7c-42a2-82aa-4c513b80fd74 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:22.427244 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8ad3af68-6e7c-42a2-82aa-4c513b80fd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:22.534190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 699/1390] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:19:22.579944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9cb9d-1030-4730-9730-017d43d63fe9 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:22.581707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b5d9cb9d-1030-4730-9730-017d43d63fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:22.625076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:22.756728 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9cb9d-1030-4730-9730-017d43d63fe9 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:22.876708 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:22.889627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5527a6-413d-47eb-8e40-110b514723e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:22.890238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5527a6-413d-47eb-8e40-110b514723e5 service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:22.890518 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5527a6-413d-47eb-8e40-110b514723e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:22.890791 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5527a6-413d-47eb-8e40-110b514723e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:22.892467 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-da5527a6-413d-47eb-8e40-110b514723e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:22.953169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b1bbb-910b-4d30-9b63-24abdd329a68 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:22.953450 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b1bbb-910b-4d30-9b63-24abdd329a68 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:22.953742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935b1bbb-910b-4d30-9b63-24abdd329a68 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:22.955392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 700/1391] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:22.960454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819e148-b14c-4302-96d4-e8e05cb119c9 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:22.960722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819e148-b14c-4302-96d4-e8e05cb119c9 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:22.961007 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819e148-b14c-4302-96d4-e8e05cb119c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:22.979065 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:22.994023 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5527a6-413d-47eb-8e40-110b514723e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:22.995680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5527a6-413d-47eb-8e40-110b514723e5 service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:23.019859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 692/1392] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:19:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:19:23.040742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-9819e148-b14c-4302-96d4-e8e05cb119c9 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:23.041989 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9819e148-b14c-4302-96d4-e8e05cb119c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:23.185268 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 701/1393] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:19:23.243053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84882172-3be9-49ed-bc25-eba09ff46d10 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:23.243375 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84882172-3be9-49ed-bc25-eba09ff46d10 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:23.243607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84882172-3be9-49ed-bc25-eba09ff46d10 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:23.245045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 693/1394] 10.222.0.17 () {54 vars in 966 bytes} [Sun Aug 7 19:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:23.251430 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6d96b-754f-406f-9ebd-a0daf060bcfd None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:23.251889 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6d96b-754f-406f-9ebd-a0daf060bcfd None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:23.252368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6d96b-754f-406f-9ebd-a0daf060bcfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:23.360309 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-8fd6d96b-754f-406f-9ebd-a0daf060bcfd None None] MFA Rules not processed for user `a9daa1a3c7654dcfa5ccb03f467ce7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:23.362294 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8fd6d96b-754f-406f-9ebd-a0daf060bcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:23.415881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:23.481214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 702/1395] 10.222.0.17 () {58 vars in 1063 bytes} [Sun Aug 7 19:19:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:19:23.490576 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84882172-3be9-49ed-bc25-eba09ff46d10 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:23.493487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-84882172-3be9-49ed-bc25-eba09ff46d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:23.641177 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84882172-3be9-49ed-bc25-eba09ff46d10 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:23.655858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:23.738612 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb3c925-d412-4fd6-a71a-4e3dedd1dadd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9daa1a3c7654dcfa5ccb03f467ce7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:23.739288 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3c925-d412-4fd6-a71a-4e3dedd1dadd service glance] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:23.739639 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3c925-d412-4fd6-a71a-4e3dedd1dadd service glance] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:23.739977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb3c925-d412-4fd6-a71a-4e3dedd1dadd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:23.742168 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8cb3c925-d412-4fd6-a71a-4e3dedd1dadd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:23.887450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3c925-d412-4fd6-a71a-4e3dedd1dadd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:23.889117 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb3c925-d412-4fd6-a71a-4e3dedd1dadd service glance] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:23.914414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 694/1396] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:19:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:19:24.974300 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e9140-a0cb-42c7-967b-9d0ee13749e1 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:24.975156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e9140-a0cb-42c7-967b-9d0ee13749e1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:24.975156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e9140-a0cb-42c7-967b-9d0ee13749e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:25.087426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-a92e9140-a0cb-42c7-967b-9d0ee13749e1 None None] MFA Rules not processed for user `61ff29258a214817b8fc15b51fe8b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:25.088795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a92e9140-a0cb-42c7-967b-9d0ee13749e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:25.132267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56f1d4-4d69-4041-bbc3-e3e1e69b7ea5 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:25.132852 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56f1d4-4d69-4041-bbc3-e3e1e69b7ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:25.133237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56f1d4-4d69-4041-bbc3-e3e1e69b7ea5 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:25.135836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 695/1397] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:19:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:25.145766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f20980-cd6f-4201-8af2-a6252859fe69 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:25.146510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f20980-cd6f-4201-8af2-a6252859fe69 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:25.146963 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f20980-cd6f-4201-8af2-a6252859fe69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:25.196867 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 703/1398] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Aug 7 19:19:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:19:25.225845 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:25.245316 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-99f20980-cd6f-4201-8af2-a6252859fe69 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:25.247077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-99f20980-cd6f-4201-8af2-a6252859fe69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:25.258693 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f6474-68c5-4340-8507-ef7a0de10505 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:25.259164 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f6474-68c5-4340-8507-ef7a0de10505 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:25.259606 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f6474-68c5-4340-8507-ef7a0de10505 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:25.261863 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 704/1399] 10.222.0.17 () {54 vars in 943 bytes} [Sun Aug 7 19:19:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:19:25.271438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3554955-22b1-4fce-87fe-985a15c3d5a8 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:25.271961 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3554955-22b1-4fce-87fe-985a15c3d5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:25.272533 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3554955-22b1-4fce-87fe-985a15c3d5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:25.384309 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:25.387109 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 696/1400] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:19:25.415555 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-e3554955-22b1-4fce-87fe-985a15c3d5a8 None None] MFA Rules not processed for user `db51971158a946c98a8ed716ecc5d72b`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:25.417701 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e3554955-22b1-4fce-87fe-985a15c3d5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:25.438322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f20980-cd6f-4201-8af2-a6252859fe69 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:25.440259 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-99f20980-cd6f-4201-8af2-a6252859fe69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:25.576939 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 705/1401] 10.222.0.17 () {58 vars in 1040 bytes} [Sun Aug 7 19:19:25 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:19:25.588573 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3554955-22b1-4fce-87fe-985a15c3d5a8 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:25.600033 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3554955-22b1-4fce-87fe-985a15c3d5a8 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:25.606272 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f20980-cd6f-4201-8af2-a6252859fe69 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:25.622282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:25.743899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:25.751220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa33f9-8f1a-406e-9cb5-4eb661eeef8f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db51971158a946c98a8ed716ecc5d72b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:25.751845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa33f9-8f1a-406e-9cb5-4eb661eeef8f service swift] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:25.752157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa33f9-8f1a-406e-9cb5-4eb661eeef8f service swift] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:25.752429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa33f9-8f1a-406e-9cb5-4eb661eeef8f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:25.754521 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a8aa33f9-8f1a-406e-9cb5-4eb661eeef8f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:25.798398 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864efd92-99e2-40ee-bc20-9b6c07a5166a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:25.799464 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864efd92-99e2-40ee-bc20-9b6c07a5166a service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:25.799955 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864efd92-99e2-40ee-bc20-9b6c07a5166a service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:25.800576 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864efd92-99e2-40ee-bc20-9b6c07a5166a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:25.803917 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-864efd92-99e2-40ee-bc20-9b6c07a5166a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:25.851423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa33f9-8f1a-406e-9cb5-4eb661eeef8f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:25.853009 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa33f9-8f1a-406e-9cb5-4eb661eeef8f service swift] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:25.862135 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 706/1402] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Aug 7 19:19:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 07 19:19:25.940284 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864efd92-99e2-40ee-bc20-9b6c07a5166a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:25.941980 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864efd92-99e2-40ee-bc20-9b6c07a5166a service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:25.964647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 697/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:19:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:19:28.395102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2b6cf-7342-4041-b830-c7fe374f0ee9 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:28.396222 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2b6cf-7342-4041-b830-c7fe374f0ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:28.396772 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2b6cf-7342-4041-b830-c7fe374f0ee9 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:28.398501 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 707/1404] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:19:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:28.404318 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b5352-f28c-4f43-9f0e-7ee0d8851d5c None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:28.405179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b5352-f28c-4f43-9f0e-7ee0d8851d5c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:28.405179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b5352-f28c-4f43-9f0e-7ee0d8851d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:28.442410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:28.528882 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-a72b5352-f28c-4f43-9f0e-7ee0d8851d5c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:28.531800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-a72b5352-f28c-4f43-9f0e-7ee0d8851d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:28.771689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 698/1405] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:19:28.818907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2b6cf-7342-4041-b830-c7fe374f0ee9 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:28.820585 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f7d2b6cf-7342-4041-b830-c7fe374f0ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:29.132941 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:29.158388 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2b6cf-7342-4041-b830-c7fe374f0ee9 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:29.415149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a7771-5786-422f-9ff3-1b68cb272da8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:29.416310 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a7771-5786-422f-9ff3-1b68cb272da8 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:29.416474 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a7771-5786-422f-9ff3-1b68cb272da8 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:29.416949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a7771-5786-422f-9ff3-1b68cb272da8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:29.423125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-712a7771-5786-422f-9ff3-1b68cb272da8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:29.558411 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a7771-5786-422f-9ff3-1b68cb272da8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:29.560060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a7771-5786-422f-9ff3-1b68cb272da8 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:29.571781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:29.581765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 708/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:19:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 763 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:19:32.385446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5b754-1e30-49e1-974b-50ee247399e3 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:32.386269 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5b754-1e30-49e1-974b-50ee247399e3 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:32.386269 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5b754-1e30-49e1-974b-50ee247399e3 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:32.388909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 699/1407] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:19:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:32.398488 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e8a9a-1926-40bb-814e-9d0f443877ad None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:32.399303 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e8a9a-1926-40bb-814e-9d0f443877ad None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:32.399477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e8a9a-1926-40bb-814e-9d0f443877ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:32.535232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-e69e8a9a-1926-40bb-814e-9d0f443877ad None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:32.537408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e69e8a9a-1926-40bb-814e-9d0f443877ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:32.695656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 709/1408] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:19:32.743518 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5b754-1e30-49e1-974b-50ee247399e3 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:32.745652 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-a5e5b754-1e30-49e1-974b-50ee247399e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:32.868541 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:32.975236 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5b754-1e30-49e1-974b-50ee247399e3 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:33.229293 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f73d9-ca9b-44b3-97b3-d55ce54cd1e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:33.230192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f73d9-ca9b-44b3-97b3-d55ce54cd1e0 service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:33.230680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f73d9-ca9b-44b3-97b3-d55ce54cd1e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:33.231167 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f73d9-ca9b-44b3-97b3-d55ce54cd1e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:33.242708 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-988f73d9-ca9b-44b3-97b3-d55ce54cd1e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:33.330772 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:33.389997 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f73d9-ca9b-44b3-97b3-d55ce54cd1e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:33.392529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f73d9-ca9b-44b3-97b3-d55ce54cd1e0 service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:33.428085 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 700/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:19:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 679 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:19:42.693850 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=67 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:43.418163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=20 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:49.823834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88885d6-062d-4129-99ed-99dbb008c3aa None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:49.824572 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88885d6-062d-4129-99ed-99dbb008c3aa None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:49.824572 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88885d6-062d-4129-99ed-99dbb008c3aa None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:49.825829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 710/1410] 10.222.0.17 () {54 vars in 918 bytes} [Sun Aug 7 19:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:49.832832 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8950a7-ff12-4efe-915a-4afad7da5afa None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:49.833397 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8950a7-ff12-4efe-915a-4afad7da5afa None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:49.833870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8950a7-ff12-4efe-915a-4afad7da5afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:49.972711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-4f8950a7-ff12-4efe-915a-4afad7da5afa None None] MFA Rules not processed for user `143c30b4201645279564e72ab3288adc`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:49.974110 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4f8950a7-ff12-4efe-915a-4afad7da5afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:50.112641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 701/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Sun Aug 7 19:19:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:19:50.129316 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06289d5-5c82-4a4e-b1e5-075c348cbbc7 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:50.129641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06289d5-5c82-4a4e-b1e5-075c348cbbc7 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:50.130025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06289d5-5c82-4a4e-b1e5-075c348cbbc7 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:50.131498 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 711/1412] 10.222.0.17 () {54 vars in 918 bytes} [Sun Aug 7 19:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:50.137905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8aa81-841b-4ed3-ad0d-08433cf559a2 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:50.138205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8aa81-841b-4ed3-ad0d-08433cf559a2 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:50.138535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8aa81-841b-4ed3-ad0d-08433cf559a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:50.237657 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:50.240872 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-94a8aa81-841b-4ed3-ad0d-08433cf559a2 None None] MFA Rules not processed for user `a4946aec299f45bd8402cff3159d04d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:50.242734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-94a8aa81-841b-4ed3-ad0d-08433cf559a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:50.379182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 702/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Sun Aug 7 19:19:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:19:52.955609 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bea740-3ba2-4ecb-bbf8-474469ef75bc None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:52.956299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bea740-3ba2-4ecb-bbf8-474469ef75bc None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:52.956428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bea740-3ba2-4ecb-bbf8-474469ef75bc None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:52.957826 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 19:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:52.964123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857d36e-3aae-4013-beb7-6c9f18e74536 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:52.964680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857d36e-3aae-4013-beb7-6c9f18e74536 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:52.964680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857d36e-3aae-4013-beb7-6c9f18e74536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:53.054360 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-4857d36e-3aae-4013-beb7-6c9f18e74536 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:53.056855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4857d36e-3aae-4013-beb7-6c9f18e74536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:53.126259 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:53.178133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 19:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:19:53.223877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bea740-3ba2-4ecb-bbf8-474469ef75bc None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:53.225688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-17bea740-3ba2-4ecb-bbf8-474469ef75bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:53.408679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bea740-3ba2-4ecb-bbf8-474469ef75bc None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:53.491638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:53.538076 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deda315a-927f-4024-b476-bfdfd65bd350 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:53.538683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda315a-927f-4024-b476-bfdfd65bd350 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:53.538974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda315a-927f-4024-b476-bfdfd65bd350 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:53.539232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deda315a-927f-4024-b476-bfdfd65bd350 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:53.540945 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-deda315a-927f-4024-b476-bfdfd65bd350 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:53.641852 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda315a-927f-4024-b476-bfdfd65bd350 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:53.644029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deda315a-927f-4024-b476-bfdfd65bd350 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:53.667437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 713/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:19:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:19:54.716837 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8137dd4-65f6-426f-b1e4-904c23152e48 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:54.717545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8137dd4-65f6-426f-b1e4-904c23152e48 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:54.717808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8137dd4-65f6-426f-b1e4-904c23152e48 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:54.720411 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 19:19:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:54.727290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01724654-4189-49e2-aef5-4d9c107f72c1 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:54.728119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01724654-4189-49e2-aef5-4d9c107f72c1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:54.728119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01724654-4189-49e2-aef5-4d9c107f72c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:54.823616 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-01724654-4189-49e2-aef5-4d9c107f72c1 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:54.824984 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-01724654-4189-49e2-aef5-4d9c107f72c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:54.855481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:54.939568 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 19:19:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:19:55.100851 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8137dd4-65f6-426f-b1e4-904c23152e48 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:55.102688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-c8137dd4-65f6-426f-b1e4-904c23152e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:55.313596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8137dd4-65f6-426f-b1e4-904c23152e48 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:55.365769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:55.484274 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f5712-fcfc-4fd1-a387-c5ac563867da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:55.484914 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f5712-fcfc-4fd1-a387-c5ac563867da service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:55.485186 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f5712-fcfc-4fd1-a387-c5ac563867da service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:55.485455 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f5712-fcfc-4fd1-a387-c5ac563867da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:55.487230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ce7f5712-fcfc-4fd1-a387-c5ac563867da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:55.587976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f5712-fcfc-4fd1-a387-c5ac563867da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:55.589689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f5712-fcfc-4fd1-a387-c5ac563867da service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:55.611423 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 705/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:19:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:19:56.536585 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe234cc3-8158-4f70-a78d-8b6c98f9ba3a None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:56.537204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe234cc3-8158-4f70-a78d-8b6c98f9ba3a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:56.537204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe234cc3-8158-4f70-a78d-8b6c98f9ba3a None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:56.538864 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 19:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:56.545297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3247d91-d3ae-4a34-86d9-bb6c0361c696 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:56.545874 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3247d91-d3ae-4a34-86d9-bb6c0361c696 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:56.545874 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3247d91-d3ae-4a34-86d9-bb6c0361c696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:56.611489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:56.637616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-a3247d91-d3ae-4a34-86d9-bb6c0361c696 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:56.639002 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-a3247d91-d3ae-4a34-86d9-bb6c0361c696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:56.748752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 19:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:19:56.972117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe234cc3-8158-4f70-a78d-8b6c98f9ba3a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:56.974389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-fe234cc3-8158-4f70-a78d-8b6c98f9ba3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:57.139353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe234cc3-8158-4f70-a78d-8b6c98f9ba3a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:57.154359 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:57.281457 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f6ae4-b2a3-4e10-bb85-0c8633690045 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:57.281980 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6ae4-b2a3-4e10-bb85-0c8633690045 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:57.282247 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6ae4-b2a3-4e10-bb85-0c8633690045 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:57.282507 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6ae4-b2a3-4e10-bb85-0c8633690045 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:57.284347 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-515f6ae4-b2a3-4e10-bb85-0c8633690045 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:57.384101 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f6ae4-b2a3-4e10-bb85-0c8633690045 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:57.385729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f6ae4-b2a3-4e10-bb85-0c8633690045 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:57.405302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 716/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:19:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:19:58.344863 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42111fe5-b175-4efb-bc8e-3faf6638ff75 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:58.345466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42111fe5-b175-4efb-bc8e-3faf6638ff75 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:58.345466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42111fe5-b175-4efb-bc8e-3faf6638ff75 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:58.346910 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 19:19:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:19:58.355083 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41351701-2e2e-4184-a8f9-2f572372c3e2 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:58.355903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41351701-2e2e-4184-a8f9-2f572372c3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:58.355903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41351701-2e2e-4184-a8f9-2f572372c3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:58.378901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:58.442316 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-41351701-2e2e-4184-a8f9-2f572372c3e2 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:19:58.443594 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-41351701-2e2e-4184-a8f9-2f572372c3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:58.549666 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 19:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:19:58.822037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42111fe5-b175-4efb-bc8e-3faf6638ff75 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:19:58.823745 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-42111fe5-b175-4efb-bc8e-3faf6638ff75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:58.944876 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:58.984546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42111fe5-b175-4efb-bc8e-3faf6638ff75 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:19:59.111952 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c990d-bb7a-43f3-b587-316d58647ac4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:19:59.112511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c990d-bb7a-43f3-b587-316d58647ac4 service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:19:59.112788 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c990d-bb7a-43f3-b587-316d58647ac4 service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:19:59.113036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c990d-bb7a-43f3-b587-316d58647ac4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:19:59.114735 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-3d6c990d-bb7a-43f3-b587-316d58647ac4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:19:59.200703 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:19:59.218170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c990d-bb7a-43f3-b587-316d58647ac4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:19:59.219954 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c990d-bb7a-43f3-b587-316d58647ac4 service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:19:59.241905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 708/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:19:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:20:00.133494 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53764452-d430-4c7f-8dbd-a73f5bf37526 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:00.134114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53764452-d430-4c7f-8dbd-a73f5bf37526 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:00.134114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53764452-d430-4c7f-8dbd-a73f5bf37526 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:00.135728 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 19:20:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:20:00.141778 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819d774-582c-40a4-9456-4e733069663b None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:00.142366 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819d774-582c-40a4-9456-4e733069663b None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:00.142366 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819d774-582c-40a4-9456-4e733069663b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:00.246265 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-e819d774-582c-40a4-9456-4e733069663b None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:20:00.247522 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e819d774-582c-40a4-9456-4e733069663b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:00.355510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 19:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:20:00.505002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53764452-d430-4c7f-8dbd-a73f5bf37526 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:20:00.506599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-53764452-d430-4c7f-8dbd-a73f5bf37526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:00.613418 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:00.702856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53764452-d430-4c7f-8dbd-a73f5bf37526 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:20:00.839518 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b21db2-18e8-47f4-8853-4671de734b82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:20:00.840238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b21db2-18e8-47f4-8853-4671de734b82 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:00.840552 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b21db2-18e8-47f4-8853-4671de734b82 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:00.840930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b21db2-18e8-47f4-8853-4671de734b82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:00.842957 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-d8b21db2-18e8-47f4-8853-4671de734b82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:00.898750 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:00.954666 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b21db2-18e8-47f4-8853-4671de734b82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:20:00.956440 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b21db2-18e8-47f4-8853-4671de734b82 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:20:00.976969 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 719/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:20:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:20:01.845076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae7a05-9f31-4be8-8a61-ce4dd050dad2 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:01.845973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae7a05-9f31-4be8-8a61-ce4dd050dad2 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:01.845973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae7a05-9f31-4be8-8a61-ce4dd050dad2 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:01.847462 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 19:20:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:20:01.853582 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be02165-fa79-4ec5-bef0-81d7f7792cf6 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:01.854169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be02165-fa79-4ec5-bef0-81d7f7792cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:01.854169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be02165-fa79-4ec5-bef0-81d7f7792cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:01.939284 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-9be02165-fa79-4ec5-bef0-81d7f7792cf6 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:20:01.940810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9be02165-fa79-4ec5-bef0-81d7f7792cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:02.051561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 19:20:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:20:02.206020 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae7a05-9f31-4be8-8a61-ce4dd050dad2 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:20:02.208920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f1ae7a05-9f31-4be8-8a61-ce4dd050dad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:02.281132 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:02.392953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae7a05-9f31-4be8-8a61-ce4dd050dad2 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:20:02.582283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a22763-03cb-498a-9875-c04eb4936d57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:20:02.583023 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a22763-03cb-498a-9875-c04eb4936d57 service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:02.583346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a22763-03cb-498a-9875-c04eb4936d57 service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:02.583647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a22763-03cb-498a-9875-c04eb4936d57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:02.585383 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-c9a22763-03cb-498a-9875-c04eb4936d57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:02.593376 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:02.687069 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a22763-03cb-498a-9875-c04eb4936d57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:20:02.688724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a22763-03cb-498a-9875-c04eb4936d57 service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:20:02.708315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 711/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:20:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:20:03.544491 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375deca-bd9e-4a69-901e-03bde034948f None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:03.545118 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375deca-bd9e-4a69-901e-03bde034948f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:03.545118 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375deca-bd9e-4a69-901e-03bde034948f None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:03.546779 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 19:20:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:20:03.553059 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f96cf-63c2-47dd-bf4b-bf94315d63c7 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:03.553613 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f96cf-63c2-47dd-bf4b-bf94315d63c7 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:03.553778 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f96cf-63c2-47dd-bf4b-bf94315d63c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:03.644606 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-441f96cf-63c2-47dd-bf4b-bf94315d63c7 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:20:03.646041 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-441f96cf-63c2-47dd-bf4b-bf94315d63c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:03.726479 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:03.765883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 19:20:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:20:03.921656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375deca-bd9e-4a69-901e-03bde034948f None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:20:03.923204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-d375deca-bd9e-4a69-901e-03bde034948f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:03.942766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:04.102581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375deca-bd9e-4a69-901e-03bde034948f None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:20:04.218854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:04.250164 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6443b60-e481-441e-9a58-06ca61c86ce6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:20:04.250789 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6443b60-e481-441e-9a58-06ca61c86ce6 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:04.251079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6443b60-e481-441e-9a58-06ca61c86ce6 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:04.251341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6443b60-e481-441e-9a58-06ca61c86ce6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:04.253184 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-d6443b60-e481-441e-9a58-06ca61c86ce6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:04.364539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6443b60-e481-441e-9a58-06ca61c86ce6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:20:04.367513 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6443b60-e481-441e-9a58-06ca61c86ce6 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:20:04.401409 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 722/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:20:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:20:05.336284 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808c3e2-58fd-4c7b-9f91-c5d2461b90f3 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:05.337145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808c3e2-58fd-4c7b-9f91-c5d2461b90f3 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:05.337145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808c3e2-58fd-4c7b-9f91-c5d2461b90f3 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:05.338862 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 19:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:20:05.345173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f10334-a2a0-4f25-ab76-a82091979c29 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:05.345901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f10334-a2a0-4f25-ab76-a82091979c29 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:05.345901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f10334-a2a0-4f25-ab76-a82091979c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:05.440769 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-31f10334-a2a0-4f25-ab76-a82091979c29 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:20:05.442227 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-31f10334-a2a0-4f25-ab76-a82091979c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:05.496679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:05.599477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 19:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:20:05.788452 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808c3e2-58fd-4c7b-9f91-c5d2461b90f3 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:20:05.791160 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e808c3e2-58fd-4c7b-9f91-c5d2461b90f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:05.987094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808c3e2-58fd-4c7b-9f91-c5d2461b90f3 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:20:06.053741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:06.127638 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b086a81-e05e-45c5-9028-2e247dd846c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:20:06.128317 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b086a81-e05e-45c5-9028-2e247dd846c6 service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:06.128599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b086a81-e05e-45c5-9028-2e247dd846c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:06.128869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b086a81-e05e-45c5-9028-2e247dd846c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:06.130597 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-3b086a81-e05e-45c5-9028-2e247dd846c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:06.239119 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b086a81-e05e-45c5-9028-2e247dd846c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:20:06.240767 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b086a81-e05e-45c5-9028-2e247dd846c6 service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:20:06.261530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 714/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:20:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:20:07.120385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062a817-5181-4580-860c-50de4044a8f6 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:07.121410 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062a817-5181-4580-860c-50de4044a8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:07.121410 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062a817-5181-4580-860c-50de4044a8f6 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:07.122281 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 19:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:20:07.128296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f698089-de68-48c2-b2b5-f7dca5709e3a None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:07.128847 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f698089-de68-48c2-b2b5-f7dca5709e3a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:07.129009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f698089-de68-48c2-b2b5-f7dca5709e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:07.229216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-5f698089-de68-48c2-b2b5-f7dca5709e3a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:20:07.230471 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5f698089-de68-48c2-b2b5-f7dca5709e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:07.233047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:07.352213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 19:20:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:20:07.506546 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062a817-5181-4580-860c-50de4044a8f6 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:20:07.508217 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a062a817-5181-4580-860c-50de4044a8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:07.700884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062a817-5181-4580-860c-50de4044a8f6 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:20:07.735964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:07.850801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b1863-dc75-4af4-9635-6ad963f61390 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:20:07.851432 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b1863-dc75-4af4-9635-6ad963f61390 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:07.851706 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b1863-dc75-4af4-9635-6ad963f61390 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:07.851971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b1863-dc75-4af4-9635-6ad963f61390 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:07.853764 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-7c7b1863-dc75-4af4-9635-6ad963f61390 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:07.963857 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b1863-dc75-4af4-9635-6ad963f61390 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:20:07.965440 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7b1863-dc75-4af4-9635-6ad963f61390 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:20:07.985218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 725/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:20:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:20:09.932861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10d2b9-697c-4df0-b6df-7eeec7de1583 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:09.933687 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10d2b9-697c-4df0-b6df-7eeec7de1583 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:09.933687 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c10d2b9-697c-4df0-b6df-7eeec7de1583 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:09.935109 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 19:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:20:09.941383 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d50e5-3450-4024-aa7b-fe52dc335691 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:09.941944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d50e5-3450-4024-aa7b-fe52dc335691 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:09.941944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d50e5-3450-4024-aa7b-fe52dc335691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:09.988350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:10.020302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-657d50e5-3450-4024-aa7b-fe52dc335691 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:20:10.021582 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-657d50e5-3450-4024-aa7b-fe52dc335691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:10.148834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 19:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:20:10.189643 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10d2b9-697c-4df0-b6df-7eeec7de1583 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:20:10.191109 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8c10d2b9-697c-4df0-b6df-7eeec7de1583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:10.309264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c10d2b9-697c-4df0-b6df-7eeec7de1583 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:20:10.344838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:10.407228 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f6b92-b537-48a4-80b8-b8c5542da269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:20:10.408418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f6b92-b537-48a4-80b8-b8c5542da269 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:10.408909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f6b92-b537-48a4-80b8-b8c5542da269 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:10.409430 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f6b92-b537-48a4-80b8-b8c5542da269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:10.412500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-dd4f6b92-b537-48a4-80b8-b8c5542da269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:10.583765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f6b92-b537-48a4-80b8-b8c5542da269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:20:10.585170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f6b92-b537-48a4-80b8-b8c5542da269 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:20:10.605365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 717/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:20:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:20:12.880250 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213839a-ccb5-45bb-a8a2-4f5265212e72 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:12.880841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213839a-ccb5-45bb-a8a2-4f5265212e72 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:12.880841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213839a-ccb5-45bb-a8a2-4f5265212e72 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:12.882195 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 19:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:20:12.888044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edccd17-fd92-4d1a-99c1-651821fb350d None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:12.888833 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edccd17-fd92-4d1a-99c1-651821fb350d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:12.888833 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edccd17-fd92-4d1a-99c1-651821fb350d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:12.963256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-2edccd17-fd92-4d1a-99c1-651821fb350d None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:20:12.964585 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-2edccd17-fd92-4d1a-99c1-651821fb350d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:12.979543 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:13.078384 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 19:20:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:20:13.121368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213839a-ccb5-45bb-a8a2-4f5265212e72 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:20:13.122897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9213839a-ccb5-45bb-a8a2-4f5265212e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:13.265717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213839a-ccb5-45bb-a8a2-4f5265212e72 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:20:13.279885 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:13.350401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00c3dd1-fb14-4d4c-be3c-4f25441590f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:20:13.351043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c3dd1-fb14-4d4c-be3c-4f25441590f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:13.351323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c3dd1-fb14-4d4c-be3c-4f25441590f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:13.351594 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00c3dd1-fb14-4d4c-be3c-4f25441590f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:13.353453 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c00c3dd1-fb14-4d4c-be3c-4f25441590f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:13.497964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00c3dd1-fb14-4d4c-be3c-4f25441590f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:20:13.499710 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00c3dd1-fb14-4d4c-be3c-4f25441590f2 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:20:13.521260 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 728/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:20:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:20:16.007455 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfc683-a9b2-48e0-85b7-41ae1e0b4f62 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:16.008310 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfc683-a9b2-48e0-85b7-41ae1e0b4f62 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:16.008310 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfc683-a9b2-48e0-85b7-41ae1e0b4f62 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:16.009451 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 19:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:20:16.015477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314a100-4e09-4d45-b3b3-73cc5c637b3c None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:16.015979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314a100-4e09-4d45-b3b3-73cc5c637b3c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:16.016099 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314a100-4e09-4d45-b3b3-73cc5c637b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:16.070348 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:16.090328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-a314a100-4e09-4d45-b3b3-73cc5c637b3c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:20:16.091571 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a314a100-4e09-4d45-b3b3-73cc5c637b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:16.197999 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 19:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:20:16.238591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfc683-a9b2-48e0-85b7-41ae1e0b4f62 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:20:16.240371 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-38cfc683-a9b2-48e0-85b7-41ae1e0b4f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:16.368996 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfc683-a9b2-48e0-85b7-41ae1e0b4f62 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:20:16.415362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:16.452759 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b3e00-b9bc-4825-b1c7-a86522f0cd1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:20:16.453384 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b3e00-b9bc-4825-b1c7-a86522f0cd1f service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:16.453665 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b3e00-b9bc-4825-b1c7-a86522f0cd1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:16.453939 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b3e00-b9bc-4825-b1c7-a86522f0cd1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:16.455600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d42b3e00-b9bc-4825-b1c7-a86522f0cd1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:16.569314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b3e00-b9bc-4825-b1c7-a86522f0cd1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:20:16.570778 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b3e00-b9bc-4825-b1c7-a86522f0cd1f service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:20:16.591680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 720/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:20:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:20:18.785760 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f98224-ff5e-484f-92c6-a6221ffa79f1 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:18.786381 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f98224-ff5e-484f-92c6-a6221ffa79f1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:18.786502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f98224-ff5e-484f-92c6-a6221ffa79f1 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:18.788145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 19:20:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:20:18.794399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442082ad-23b0-4b16-bff1-4536bf215056 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:18.794929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442082ad-23b0-4b16-bff1-4536bf215056 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:18.795062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442082ad-23b0-4b16-bff1-4536bf215056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:18.870079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-442082ad-23b0-4b16-bff1-4536bf215056 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:20:18.871326 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-442082ad-23b0-4b16-bff1-4536bf215056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:18.894280 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:18.980619 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 19:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:20:19.022893 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f98224-ff5e-484f-92c6-a6221ffa79f1 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:20:19.024754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-03f98224-ff5e-484f-92c6-a6221ffa79f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:19.141928 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f98224-ff5e-484f-92c6-a6221ffa79f1 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:20:19.170796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:19.239712 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b7016-ab72-4091-b5b0-d533f356f23c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:20:19.240372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b7016-ab72-4091-b5b0-d533f356f23c service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:19.240649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b7016-ab72-4091-b5b0-d533f356f23c service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:19.240921 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b7016-ab72-4091-b5b0-d533f356f23c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:19.242610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-731b7016-ab72-4091-b5b0-d533f356f23c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:19.374007 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b7016-ab72-4091-b5b0-d533f356f23c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:20:19.375495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b7016-ab72-4091-b5b0-d533f356f23c service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:20:19.397350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 731/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:20:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:20:22.013744 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bd632-c5b6-43e1-a508-fac11e437240 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:22.014391 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bd632-c5b6-43e1-a508-fac11e437240 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:22.014391 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bd632-c5b6-43e1-a508-fac11e437240 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:22.016086 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 19:20:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:20:22.023198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b00f3-7eb2-4c12-959a-d270e193206b None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:22.023738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b00f3-7eb2-4c12-959a-d270e193206b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:22.023853 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b00f3-7eb2-4c12-959a-d270e193206b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:22.096437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:22.100413 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-e19b00f3-7eb2-4c12-959a-d270e193206b None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:20:22.102216 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e19b00f3-7eb2-4c12-959a-d270e193206b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:22.213084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 19:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:20:22.278669 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bd632-c5b6-43e1-a508-fac11e437240 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:20:22.280480 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-c74bd632-c5b6-43e1-a508-fac11e437240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:22.394984 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bd632-c5b6-43e1-a508-fac11e437240 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:20:22.449782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:22.477980 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece3195-521d-43d2-aa6f-6f9a4e34caa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:20:22.478665 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece3195-521d-43d2-aa6f-6f9a4e34caa3 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:22.478984 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece3195-521d-43d2-aa6f-6f9a4e34caa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:22.479296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece3195-521d-43d2-aa6f-6f9a4e34caa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:22.481079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8ece3195-521d-43d2-aa6f-6f9a4e34caa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:22.605656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece3195-521d-43d2-aa6f-6f9a4e34caa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:20:22.607216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece3195-521d-43d2-aa6f-6f9a4e34caa3 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:20:22.627809 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 723/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:20:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:20:24.745141 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0790a77-4562-4265-9f8f-f708b1bf0190 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:24.745726 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0790a77-4562-4265-9f8f-f708b1bf0190 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:24.745726 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0790a77-4562-4265-9f8f-f708b1bf0190 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:24.747082 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 19:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:20:24.752787 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc2aae-882c-4773-9e83-196868b66a00 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:24.753371 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc2aae-882c-4773-9e83-196868b66a00 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:24.753371 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc2aae-882c-4773-9e83-196868b66a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:24.826417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-21cc2aae-882c-4773-9e83-196868b66a00 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:20:24.827623 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-21cc2aae-882c-4773-9e83-196868b66a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:24.857574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:24.930507 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 19:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:20:24.970400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0790a77-4562-4265-9f8f-f708b1bf0190 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:20:24.971962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f0790a77-4562-4265-9f8f-f708b1bf0190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:25.086426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0790a77-4562-4265-9f8f-f708b1bf0190 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:20:25.122413 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:25.173237 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d71c4-2c33-4fa2-bb7a-d264eb558159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:20:25.173938 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d71c4-2c33-4fa2-bb7a-d264eb558159 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:25.174219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d71c4-2c33-4fa2-bb7a-d264eb558159 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:25.174492 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d71c4-2c33-4fa2-bb7a-d264eb558159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:25.176168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9e0d71c4-2c33-4fa2-bb7a-d264eb558159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:25.299696 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d71c4-2c33-4fa2-bb7a-d264eb558159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:20:25.301218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d71c4-2c33-4fa2-bb7a-d264eb558159 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:20:25.322249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 734/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:20:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:20:27.539392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd6c86-2655-48a8-82e4-048cae5abadb None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:27.540331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd6c86-2655-48a8-82e4-048cae5abadb None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:27.540492 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd6c86-2655-48a8-82e4-048cae5abadb None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:27.542031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 19:20:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:20:27.551667 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7882f8-5d91-45ca-bc92-2b7c2301ec5e None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:27.552308 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7882f8-5d91-45ca-bc92-2b7c2301ec5e None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:27.552683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7882f8-5d91-45ca-bc92-2b7c2301ec5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:27.646713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-1d7882f8-5d91-45ca-bc92-2b7c2301ec5e None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:20:27.648322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-1d7882f8-5d91-45ca-bc92-2b7c2301ec5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:27.660084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:27.763291 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 19:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:20:27.803093 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd6c86-2655-48a8-82e4-048cae5abadb None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:20:27.805139 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-7dfd6c86-2655-48a8-82e4-048cae5abadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:27.949606 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd6c86-2655-48a8-82e4-048cae5abadb None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:20:28.023970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:28.047482 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9aec5-7e38-43f0-b497-6028ad8ac180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:20:28.048064 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9aec5-7e38-43f0-b497-6028ad8ac180 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:28.048468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9aec5-7e38-43f0-b497-6028ad8ac180 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:28.048745 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9aec5-7e38-43f0-b497-6028ad8ac180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:28.050480 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-3df9aec5-7e38-43f0-b497-6028ad8ac180 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:28.192528 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9aec5-7e38-43f0-b497-6028ad8ac180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:20:28.194123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9aec5-7e38-43f0-b497-6028ad8ac180 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:20:28.215621 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 726/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:20:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:20:30.546823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa982c9f-ad9d-469e-8233-40da2a6cc787 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:30.547464 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa982c9f-ad9d-469e-8233-40da2a6cc787 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:30.547464 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa982c9f-ad9d-469e-8233-40da2a6cc787 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:30.548892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 19:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:20:30.555190 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44812d52-da6d-495e-9e0b-79ccf597f319 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:30.555780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44812d52-da6d-495e-9e0b-79ccf597f319 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:30.555780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44812d52-da6d-495e-9e0b-79ccf597f319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:30.631576 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-44812d52-da6d-495e-9e0b-79ccf597f319 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:20:30.632858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-44812d52-da6d-495e-9e0b-79ccf597f319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:30.676849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:30.744769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 19:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:20:30.784919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa982c9f-ad9d-469e-8233-40da2a6cc787 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:20:30.786485 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-fa982c9f-ad9d-469e-8233-40da2a6cc787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:30.908081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa982c9f-ad9d-469e-8233-40da2a6cc787 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:20:30.951427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:30.992895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f5d2a-80b4-43bf-8432-cf83ddb52fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:20:30.993522 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f5d2a-80b4-43bf-8432-cf83ddb52fea service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:30.993798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f5d2a-80b4-43bf-8432-cf83ddb52fea service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:30.994074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f5d2a-80b4-43bf-8432-cf83ddb52fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:30.995727 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c66f5d2a-80b4-43bf-8432-cf83ddb52fea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:31.117993 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f5d2a-80b4-43bf-8432-cf83ddb52fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:20:31.119550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66f5d2a-80b4-43bf-8432-cf83ddb52fea service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:20:31.141201 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 737/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:20:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:20:33.239948 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195625d-2a3d-4b7f-8988-ee3fc3c87202 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:33.240678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195625d-2a3d-4b7f-8988-ee3fc3c87202 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:33.240678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195625d-2a3d-4b7f-8988-ee3fc3c87202 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:33.241920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 19:20:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:20:33.248327 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2d4cc-6675-401c-973b-87196e4f6d88 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:33.248953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2d4cc-6675-401c-973b-87196e4f6d88 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:33.248953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2d4cc-6675-401c-973b-87196e4f6d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:33.325403 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-04b2d4cc-6675-401c-973b-87196e4f6d88 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:20:33.326619 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-04b2d4cc-6675-401c-973b-87196e4f6d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:33.347976 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:33.488919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 19:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:20:33.525952 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195625d-2a3d-4b7f-8988-ee3fc3c87202 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:20:33.527553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-3195625d-2a3d-4b7f-8988-ee3fc3c87202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:33.656428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195625d-2a3d-4b7f-8988-ee3fc3c87202 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:20:33.735553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:33.747246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b556f-addb-4164-af8e-e7d595d27e37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:20:33.748319 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b556f-addb-4164-af8e-e7d595d27e37 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:33.748752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b556f-addb-4164-af8e-e7d595d27e37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:33.749092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b556f-addb-4164-af8e-e7d595d27e37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:33.751957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ce0b556f-addb-4164-af8e-e7d595d27e37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:33.889045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b556f-addb-4164-af8e-e7d595d27e37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:20:33.890537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b556f-addb-4164-af8e-e7d595d27e37 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:20:33.912198 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 729/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:20:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:20:36.033290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e222c-044d-40e5-9b2d-d3d6e1df337d None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:36.033911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e222c-044d-40e5-9b2d-d3d6e1df337d None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:36.033911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e222c-044d-40e5-9b2d-d3d6e1df337d None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:36.035482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 19:20:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:20:36.041772 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9118e4-6a34-47b7-abc7-b37488a7f7f9 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:36.042371 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9118e4-6a34-47b7-abc7-b37488a7f7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:36.042371 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9118e4-6a34-47b7-abc7-b37488a7f7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:36.154942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-2b9118e4-6a34-47b7-abc7-b37488a7f7f9 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:20:36.156960 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-2b9118e4-6a34-47b7-abc7-b37488a7f7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:36.235247 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:36.289600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 19:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:20:36.331236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e222c-044d-40e5-9b2d-d3d6e1df337d None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:20:36.332892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-923e222c-044d-40e5-9b2d-d3d6e1df337d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:36.454097 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e222c-044d-40e5-9b2d-d3d6e1df337d None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:20:36.516886 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:36.545773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163ab35-756b-4dec-a930-05ca718bf9f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:20:36.546291 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163ab35-756b-4dec-a930-05ca718bf9f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:36.546563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163ab35-756b-4dec-a930-05ca718bf9f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:36.546830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163ab35-756b-4dec-a930-05ca718bf9f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:36.548541 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8163ab35-756b-4dec-a930-05ca718bf9f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:36.685225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163ab35-756b-4dec-a930-05ca718bf9f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:20:36.686902 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163ab35-756b-4dec-a930-05ca718bf9f9 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:20:36.708961 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 740/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:20:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:20:46.287603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=22 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:46.706202 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=66 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:56.880413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f84e40-f4ee-4561-8fbb-927a1e17be35 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:56.881247 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f84e40-f4ee-4561-8fbb-927a1e17be35 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:56.881247 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f84e40-f4ee-4561-8fbb-927a1e17be35 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:56.882324 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 731/1471] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:20:56.889177 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ccd3d-6775-4254-8ca0-a523adc420ba None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:56.889767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ccd3d-6775-4254-8ca0-a523adc420ba None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:56.890220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ccd3d-6775-4254-8ca0-a523adc420ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:57.018211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-b27ccd3d-6775-4254-8ca0-a523adc420ba None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:20:57.020390 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b27ccd3d-6775-4254-8ca0-a523adc420ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:57.210229 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 741/1472] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:20:57.307284 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60343b8-7082-4ccc-9e37-c6e947eed8cc None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:57.307565 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60343b8-7082-4ccc-9e37-c6e947eed8cc None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:57.307834 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60343b8-7082-4ccc-9e37-c6e947eed8cc None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:57.309327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 732/1473] 10.222.0.17 () {54 vars in 966 bytes} [Sun Aug 7 19:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:20:57.314493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c5861-72ec-4eef-a545-d414e1ae3a0a None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:57.314776 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c5861-72ec-4eef-a545-d414e1ae3a0a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:57.315057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c5861-72ec-4eef-a545-d414e1ae3a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:57.398008 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-732c5861-72ec-4eef-a545-d414e1ae3a0a None None] MFA Rules not processed for user `a9daa1a3c7654dcfa5ccb03f467ce7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:20:57.399259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-732c5861-72ec-4eef-a545-d414e1ae3a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:57.422634 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:57.484274 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 742/1474] 10.222.0.17 () {58 vars in 1063 bytes} [Sun Aug 7 19:20:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:20:57.491942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60343b8-7082-4ccc-9e37-c6e947eed8cc None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:20:57.493523 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b60343b8-7082-4ccc-9e37-c6e947eed8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:57.649912 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60343b8-7082-4ccc-9e37-c6e947eed8cc None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:20:57.759531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59fa2f-2f41-4e7c-9fb8-be6912b9bb42 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9daa1a3c7654dcfa5ccb03f467ce7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:20:57.760098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59fa2f-2f41-4e7c-9fb8-be6912b9bb42 service glance] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:20:57.760383 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59fa2f-2f41-4e7c-9fb8-be6912b9bb42 service glance] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:20:57.760657 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59fa2f-2f41-4e7c-9fb8-be6912b9bb42 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:20:57.762354 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-af59fa2f-2f41-4e7c-9fb8-be6912b9bb42 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:20:57.825346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:20:57.888247 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59fa2f-2f41-4e7c-9fb8-be6912b9bb42 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:20:57.890698 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59fa2f-2f41-4e7c-9fb8-be6912b9bb42 service glance] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:20:57.925015 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 733/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:20:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:21:00.240690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5282df-d8c7-4fa9-b86e-040b5d178c3b None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:00.241315 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5282df-d8c7-4fa9-b86e-040b5d178c3b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:00.241315 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5282df-d8c7-4fa9-b86e-040b5d178c3b None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:00.242591 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 743/1476] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:21:00.248675 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1ef9-7549-423b-b004-bacba00b1f3c None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:00.249484 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1ef9-7549-423b-b004-bacba00b1f3c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:00.249484 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a1ef9-7549-423b-b004-bacba00b1f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:00.328014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-ce3a1ef9-7549-423b-b004-bacba00b1f3c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:21:00.329399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ce3a1ef9-7549-423b-b004-bacba00b1f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:21:00.434509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 734/1477] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:21:00.458607 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5282df-d8c7-4fa9-b86e-040b5d178c3b None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:21:00.460374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-ac5282df-d8c7-4fa9-b86e-040b5d178c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:21:00.601351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5282df-d8c7-4fa9-b86e-040b5d178c3b None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:21:00.691104 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:21:00.774719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda6bc8-0918-4cf0-9e49-aada841457c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:21:00.775759 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda6bc8-0918-4cf0-9e49-aada841457c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:00.776273 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda6bc8-0918-4cf0-9e49-aada841457c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:00.776756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda6bc8-0918-4cf0-9e49-aada841457c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:00.777901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda6bc8-0918-4cf0-9e49-aada841457c4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:21:00.781145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: WARNING py.warnings [None req-4cda6bc8-0918-4cf0-9e49-aada841457c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 19:21:00.781145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: warnings.warn(msg) Aug 07 19:21:00.781145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: Aug 07 19:21:00.783433 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda6bc8-0918-4cf0-9e49-aada841457c4 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:21:00.826938 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 744/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Sun Aug 7 19:21:00 2022] GET /identity/v3/projects => generated 2553 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:21:02.918194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a96df5-217a-48d8-8701-7a9b73920e6a None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:02.918786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a96df5-217a-48d8-8701-7a9b73920e6a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:02.918786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a96df5-217a-48d8-8701-7a9b73920e6a None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:02.920167 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 735/1479] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:21:02.926232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2464907-c930-49f6-bb8c-7f0004f226bc None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:02.926814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2464907-c930-49f6-bb8c-7f0004f226bc None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:02.926814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2464907-c930-49f6-bb8c-7f0004f226bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:03.001106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-d2464907-c930-49f6-bb8c-7f0004f226bc None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:21:03.002362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-d2464907-c930-49f6-bb8c-7f0004f226bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:21:03.150620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 745/1480] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:21:03.201055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a96df5-217a-48d8-8701-7a9b73920e6a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:21:03.202670 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-40a96df5-217a-48d8-8701-7a9b73920e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:21:03.213372 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:21:03.407519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a96df5-217a-48d8-8701-7a9b73920e6a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:21:03.507788 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:21:03.548591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d459d91-f38f-444c-ba99-1a3823dfed82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:21:03.549223 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d459d91-f38f-444c-ba99-1a3823dfed82 service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:03.549528 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d459d91-f38f-444c-ba99-1a3823dfed82 service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:03.549799 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d459d91-f38f-444c-ba99-1a3823dfed82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:03.551506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-7d459d91-f38f-444c-ba99-1a3823dfed82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:21:03.652778 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d459d91-f38f-444c-ba99-1a3823dfed82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:21:03.654410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d459d91-f38f-444c-ba99-1a3823dfed82 service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:21:03.674565 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 736/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:21:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:21:05.965035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b32d44-c9a7-446b-b526-f4cee6ac088a None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:05.965644 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b32d44-c9a7-446b-b526-f4cee6ac088a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:05.965644 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b32d44-c9a7-446b-b526-f4cee6ac088a None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:05.967164 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 746/1482] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:21:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:21:05.973112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434fc41-ab5a-4049-a6e5-c9233b312295 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:05.973680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434fc41-ab5a-4049-a6e5-c9233b312295 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:05.973793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434fc41-ab5a-4049-a6e5-c9233b312295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:06.049702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-4434fc41-ab5a-4049-a6e5-c9233b312295 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:21:06.050970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4434fc41-ab5a-4049-a6e5-c9233b312295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:21:06.103014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:21:06.165384 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 737/1483] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:21:06.203314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b32d44-c9a7-446b-b526-f4cee6ac088a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:21:06.205026 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-14b32d44-c9a7-446b-b526-f4cee6ac088a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:21:06.247349 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:21:06.343026 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b32d44-c9a7-446b-b526-f4cee6ac088a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:21:06.426702 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921dcda-7e84-4fcf-8199-7b06617b9f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:21:06.427458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921dcda-7e84-4fcf-8199-7b06617b9f4c service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:06.428220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921dcda-7e84-4fcf-8199-7b06617b9f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:06.428590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921dcda-7e84-4fcf-8199-7b06617b9f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:06.430417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-1921dcda-7e84-4fcf-8199-7b06617b9f4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:21:06.513125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:21:06.554110 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921dcda-7e84-4fcf-8199-7b06617b9f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:21:06.555567 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921dcda-7e84-4fcf-8199-7b06617b9f4c service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:21:06.577780 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 747/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:21:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:21:08.690609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3622074-8c58-4dc5-b46c-b969d6f5103b None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:08.691513 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3622074-8c58-4dc5-b46c-b969d6f5103b None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:08.691513 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3622074-8c58-4dc5-b46c-b969d6f5103b None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:08.693118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 738/1485] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:21:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:21:08.699351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c47cb-efef-493d-9a5a-c2e1d1ca3a77 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:08.699953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c47cb-efef-493d-9a5a-c2e1d1ca3a77 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:08.699953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c47cb-efef-493d-9a5a-c2e1d1ca3a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:08.783714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-3a7c47cb-efef-493d-9a5a-c2e1d1ca3a77 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:21:08.785035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3a7c47cb-efef-493d-9a5a-c2e1d1ca3a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:21:08.931656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 748/1486] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:21:08.970271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3622074-8c58-4dc5-b46c-b969d6f5103b None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:21:08.971835 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d3622074-8c58-4dc5-b46c-b969d6f5103b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:21:09.115375 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3622074-8c58-4dc5-b46c-b969d6f5103b None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:21:09.243351 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:21:09.246240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627dffc-7f84-4503-a880-da8fe766318f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:21:09.247025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627dffc-7f84-4503-a880-da8fe766318f service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:09.247412 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627dffc-7f84-4503-a880-da8fe766318f service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:09.247790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627dffc-7f84-4503-a880-da8fe766318f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:09.251060 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d627dffc-7f84-4503-a880-da8fe766318f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:21:09.389116 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627dffc-7f84-4503-a880-da8fe766318f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:21:09.390602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d627dffc-7f84-4503-a880-da8fe766318f service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:21:09.412299 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 739/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:21:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:21:11.725929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5cf0e-cb33-4352-aed4-ee2ad50dd63d None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:11.726614 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5cf0e-cb33-4352-aed4-ee2ad50dd63d None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:11.726614 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5cf0e-cb33-4352-aed4-ee2ad50dd63d None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:11.727870 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 749/1488] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:21:11.735114 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d961bf0-f14c-4244-847b-5f82e80d51b1 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:11.735701 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d961bf0-f14c-4244-847b-5f82e80d51b1 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:11.736061 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d961bf0-f14c-4244-847b-5f82e80d51b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:11.817697 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-0d961bf0-f14c-4244-847b-5f82e80d51b1 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:21:11.819083 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-0d961bf0-f14c-4244-847b-5f82e80d51b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:21:11.880436 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:21:11.931920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 740/1489] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:21:11.997620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5cf0e-cb33-4352-aed4-ee2ad50dd63d None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:21:11.999179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-81e5cf0e-cb33-4352-aed4-ee2ad50dd63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:21:12.042156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:21:12.116446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5cf0e-cb33-4352-aed4-ee2ad50dd63d None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:21:12.199943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec2734-edca-4e95-95ef-01640ac9865c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:21:12.200594 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec2734-edca-4e95-95ef-01640ac9865c service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:12.200865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec2734-edca-4e95-95ef-01640ac9865c service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:12.201131 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec2734-edca-4e95-95ef-01640ac9865c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:12.202784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-bbec2734-edca-4e95-95ef-01640ac9865c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:21:12.294390 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:21:12.328622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec2734-edca-4e95-95ef-01640ac9865c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:21:12.330112 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec2734-edca-4e95-95ef-01640ac9865c service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:21:12.350713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 750/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:21:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:21:14.747175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae647f-92dd-4ad0-aebb-7a19f56224d3 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:14.747754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae647f-92dd-4ad0-aebb-7a19f56224d3 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:14.747754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae647f-92dd-4ad0-aebb-7a19f56224d3 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:14.749130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 741/1491] 10.222.0.17 () {54 vars in 927 bytes} [Sun Aug 7 19:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:21:14.755253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fae61-7d8e-412d-833d-07036a4c3632 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:14.755832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fae61-7d8e-412d-833d-07036a4c3632 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:14.756006 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fae61-7d8e-412d-833d-07036a4c3632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:14.831564 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-e94fae61-7d8e-412d-833d-07036a4c3632 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:21:14.832823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e94fae61-7d8e-412d-833d-07036a4c3632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:21:14.937673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 751/1492] 10.222.0.17 () {58 vars in 1024 bytes} [Sun Aug 7 19:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:21:14.978021 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae647f-92dd-4ad0-aebb-7a19f56224d3 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:21:14.979747 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6aae647f-92dd-4ad0-aebb-7a19f56224d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:21:15.106728 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae647f-92dd-4ad0-aebb-7a19f56224d3 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:21:15.202273 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b9a61-1d97-4411-bc34-7d8224134bf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:21:15.202901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b9a61-1d97-4411-bc34-7d8224134bf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:21:15.203353 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b9a61-1d97-4411-bc34-7d8224134bf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:21:15.203622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b9a61-1d97-4411-bc34-7d8224134bf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:21:15.205345 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-489b9a61-1d97-4411-bc34-7d8224134bf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:21:15.215811 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:21:15.327483 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b9a61-1d97-4411-bc34-7d8224134bf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:21:15.329078 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b9a61-1d97-4411-bc34-7d8224134bf7 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:21:15.351729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 742/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:21:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:21:24.935631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=82 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:21:25.349327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=50 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:22:32.871452 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bc495-f018-4a42-ac46-dbe8e687951f None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:22:32.872103 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bc495-f018-4a42-ac46-dbe8e687951f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:22:32.872103 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bc495-f018-4a42-ac46-dbe8e687951f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:22:32.952217 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-297bc495-f018-4a42-ac46-dbe8e687951f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:22:32.953477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-297bc495-f018-4a42-ac46-dbe8e687951f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:22:33.058573 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 752/1494] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:22:33.067285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ea4ac-1af9-4ea2-a6fa-533c07122cb6 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:22:33.067842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ea4ac-1af9-4ea2-a6fa-533c07122cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:22:33.067842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ea4ac-1af9-4ea2-a6fa-533c07122cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:22:33.152971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-8b3ea4ac-1af9-4ea2-a6fa-533c07122cb6 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:22:33.154137 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8b3ea4ac-1af9-4ea2-a6fa-533c07122cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:22:33.255632 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 743/1495] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:22:33.260896 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bc495-f018-4a42-ac46-dbe8e687951f None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:22:33.262153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-297bc495-f018-4a42-ac46-dbe8e687951f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:22:33.274307 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bc495-f018-4a42-ac46-dbe8e687951f None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:22:33.338967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:22:33.374444 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7e600-3e4f-417a-9b97-77f39b11ebf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:22:33.375061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7e600-3e4f-417a-9b97-77f39b11ebf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:22:33.375425 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7e600-3e4f-417a-9b97-77f39b11ebf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:22:33.375720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7e600-3e4f-417a-9b97-77f39b11ebf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:22:33.376378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7e600-3e4f-417a-9b97-77f39b11ebf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:22:33.376677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7e600-3e4f-417a-9b97-77f39b11ebf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:22:33.377713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: WARNING py.warnings [None req-adb7e600-3e4f-417a-9b97-77f39b11ebf3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 19:22:33.377713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: warnings.warn(msg) Aug 07 19:22:33.377713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: Aug 07 19:22:33.378665 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb7e600-3e4f-417a-9b97-77f39b11ebf3 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:22:33.390376 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 753/1496] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:22:33.399011 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ea4ac-1af9-4ea2-a6fa-533c07122cb6 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:22:33.409792 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ea4ac-1af9-4ea2-a6fa-533c07122cb6 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:22:33.474498 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:22:33.510437 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1ff51-6a96-411f-8eb5-02d95fee455b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:22:33.511060 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1ff51-6a96-411f-8eb5-02d95fee455b admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:22:33.511340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1ff51-6a96-411f-8eb5-02d95fee455b admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:22:33.511607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1ff51-6a96-411f-8eb5-02d95fee455b admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:22:33.512267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1ff51-6a96-411f-8eb5-02d95fee455b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:22:33.514007 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: WARNING py.warnings [None req-26c1ff51-6a96-411f-8eb5-02d95fee455b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 19:22:33.514007 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: warnings.warn(msg) Aug 07 19:22:33.514007 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: Aug 07 19:22:33.515212 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1ff51-6a96-411f-8eb5-02d95fee455b admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:22:33.576014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 744/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:22:33 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:22:33.580385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7e600-3e4f-417a-9b97-77f39b11ebf3 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:22:33.581964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-adb7e600-3e4f-417a-9b97-77f39b11ebf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:22:33.616341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb7e600-3e4f-417a-9b97-77f39b11ebf3 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:22:33.719823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147054f-6585-4996-b775-8992073d33e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:22:33.720472 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147054f-6585-4996-b775-8992073d33e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:22:33.720743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147054f-6585-4996-b775-8992073d33e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:22:33.721007 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7147054f-6585-4996-b775-8992073d33e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:22:33.721610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7147054f-6585-4996-b775-8992073d33e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:22:33.722636 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: WARNING py.warnings [None req-7147054f-6585-4996-b775-8992073d33e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 19:22:33.722636 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: warnings.warn(msg) Aug 07 19:22:33.722636 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: Aug 07 19:22:33.723855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7147054f-6585-4996-b775-8992073d33e9 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:22:33.767489 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 754/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:22:33 2022] POST /identity/v3/users => generated 290 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:22:33.771726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1ff51-6a96-411f-8eb5-02d95fee455b admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:22:33.773370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-26c1ff51-6a96-411f-8eb5-02d95fee455b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:22:33.837208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1ff51-6a96-411f-8eb5-02d95fee455b admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:22:33.923864 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:22:33.936947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43f0c8-c054-4e2b-9548-c1e709a20a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:22:33.937752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43f0c8-c054-4e2b-9548-c1e709a20a3b admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:22:33.938034 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43f0c8-c054-4e2b-9548-c1e709a20a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:22:33.938309 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43f0c8-c054-4e2b-9548-c1e709a20a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:22:33.939036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43f0c8-c054-4e2b-9548-c1e709a20a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:22:33.940478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: WARNING py.warnings [None req-ec43f0c8-c054-4e2b-9548-c1e709a20a3b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 19:22:33.940478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: warnings.warn(msg) Aug 07 19:22:33.940478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: Aug 07 19:22:33.941386 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43f0c8-c054-4e2b-9548-c1e709a20a3b admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:22:33.954368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 745/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:22:33.958702 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147054f-6585-4996-b775-8992073d33e9 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:22:33.960308 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-7147054f-6585-4996-b775-8992073d33e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:22:33.974856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7147054f-6585-4996-b775-8992073d33e9 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:22:34.009372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:22:34.093587 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e118b60-8b23-4b36-a800-4a734d2c4b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:22:34.094325 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e118b60-8b23-4b36-a800-4a734d2c4b2e admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:22:34.094635 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e118b60-8b23-4b36-a800-4a734d2c4b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:22:34.094953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e118b60-8b23-4b36-a800-4a734d2c4b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:22:34.095735 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e118b60-8b23-4b36-a800-4a734d2c4b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:22:34.096970 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: WARNING py.warnings [None req-1e118b60-8b23-4b36-a800-4a734d2c4b2e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 19:22:34.096970 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: warnings.warn(msg) Aug 07 19:22:34.096970 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: Aug 07 19:22:34.098025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e118b60-8b23-4b36-a800-4a734d2c4b2e admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:22:34.114259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 755/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:22:34.118755 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43f0c8-c054-4e2b-9548-c1e709a20a3b admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:22:34.120364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ec43f0c8-c054-4e2b-9548-c1e709a20a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:22:34.156149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43f0c8-c054-4e2b-9548-c1e709a20a3b admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:22:34.252614 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef783393-99a9-4472-802a-ec5e435e4514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:22:34.253210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef783393-99a9-4472-802a-ec5e435e4514 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:22:34.253489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef783393-99a9-4472-802a-ec5e435e4514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:22:34.253789 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef783393-99a9-4472-802a-ec5e435e4514 admin admin] PATH_INFO: `/v3/projects/c32036a7352f4e3da4548f522ccc5bca/users/68630fd3d92240d0a6fa5353366799d2/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:22:34.279981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef783393-99a9-4472-802a-ec5e435e4514 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c32036a7352f4e3da4548f522ccc5bca, user_id=68630fd3d92240d0a6fa5353366799d2, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:22:34.281723 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: WARNING py.warnings [None req-ef783393-99a9-4472-802a-ec5e435e4514 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 19:22:34.281723 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: warnings.warn(msg) Aug 07 19:22:34.281723 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: Aug 07 19:22:34.285306 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef783393-99a9-4472-802a-ec5e435e4514 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:22:34.320710 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 746/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:22:34 2022] PUT /identity/v3/projects/c32036a7352f4e3da4548f522ccc5bca/users/68630fd3d92240d0a6fa5353366799d2/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:22:34.333081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f3483-a28d-4bb7-9a8d-528d68354c4c None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:22:34.333346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f3483-a28d-4bb7-9a8d-528d68354c4c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:22:34.333651 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f3483-a28d-4bb7-9a8d-528d68354c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:22:34.396212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-410f3483-a28d-4bb7-9a8d-528d68354c4c None None] MFA Rules not processed for user `68630fd3d92240d0a6fa5353366799d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:22:34.397431 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-410f3483-a28d-4bb7-9a8d-528d68354c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:22:34.479827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 756/1502] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:22:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:22:34.493552 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e8031-a137-4ce2-90ab-946d39bab57f None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:22:34.493839 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e8031-a137-4ce2-90ab-946d39bab57f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:22:34.494115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e8031-a137-4ce2-90ab-946d39bab57f None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:22:34.495500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 747/1503] 10.222.0.17 () {54 vars in 964 bytes} [Sun Aug 7 19:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:22:34.500585 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73793f11-9d54-48b0-a1ac-6b7e909d9244 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:22:34.500871 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73793f11-9d54-48b0-a1ac-6b7e909d9244 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:22:34.501149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73793f11-9d54-48b0-a1ac-6b7e909d9244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:22:34.535529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:22:34.583742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-73793f11-9d54-48b0-a1ac-6b7e909d9244 None None] MFA Rules not processed for user `f83e5f0b1d5e4f57bbf283d420f080ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:22:34.585018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-73793f11-9d54-48b0-a1ac-6b7e909d9244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:22:34.668732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 757/1504] 10.222.0.17 () {58 vars in 1061 bytes} [Sun Aug 7 19:22:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:22:34.677819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e8031-a137-4ce2-90ab-946d39bab57f None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:22:34.679507 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4d3e8031-a137-4ce2-90ab-946d39bab57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:22:34.740733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:22:34.803096 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e8031-a137-4ce2-90ab-946d39bab57f None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:22:34.886691 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f51110-e838-4068-9cbb-14feb3942a7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:22:34.887322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f51110-e838-4068-9cbb-14feb3942a7d service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:22:34.887599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f51110-e838-4068-9cbb-14feb3942a7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:22:34.887901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f51110-e838-4068-9cbb-14feb3942a7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:22:34.889598 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f9f51110-e838-4068-9cbb-14feb3942a7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:22:35.003373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f51110-e838-4068-9cbb-14feb3942a7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:22:35.004849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f51110-e838-4068-9cbb-14feb3942a7d service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:22:35.008392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:22:35.023657 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 748/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:22:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:22:35.073838 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40302c76-107a-4bc5-9d4a-7df4a6d2652c None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:22:35.074302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40302c76-107a-4bc5-9d4a-7df4a6d2652c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:22:35.074673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40302c76-107a-4bc5-9d4a-7df4a6d2652c None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:22:35.076535 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 758/1506] 10.222.0.17 () {52 vars in 830 bytes} [Sun Aug 7 19:22:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:22:35.081538 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f51110-e838-4068-9cbb-14feb3942a7d service cinder] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:22:35.082798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f9f51110-e838-4068-9cbb-14feb3942a7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:22:35.190658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f51110-e838-4068-9cbb-14feb3942a7d service cinder] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:22:35.289419 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baefa1-8167-4484-a017-c9e7ad7bca68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:22:35.290064 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baefa1-8167-4484-a017-c9e7ad7bca68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:22:35.290348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baefa1-8167-4484-a017-c9e7ad7bca68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:22:35.290622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8baefa1-8167-4484-a017-c9e7ad7bca68 admin admin] PATH_INFO: `/v3/users/68630fd3d92240d0a6fa5353366799d2` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:22:35.304478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8baefa1-8167-4484-a017-c9e7ad7bca68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68630fd3d92240d0a6fa5353366799d2)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:22:35.305459 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: WARNING py.warnings [None req-b8baefa1-8167-4484-a017-c9e7ad7bca68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 19:22:35.305459 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: warnings.warn(msg) Aug 07 19:22:35.305459 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: Aug 07 19:22:35.306683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8baefa1-8167-4484-a017-c9e7ad7bca68 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:22:35.317017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:22:35.375451 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-b8baefa1-8167-4484-a017-c9e7ad7bca68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68630fd3d92240d0a6fa5353366799d2'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:22:35.399964 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-b8baefa1-8167-4484-a017-c9e7ad7bca68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68630fd3d92240d0a6fa5353366799d2'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:22:35.415087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-b8baefa1-8167-4484-a017-c9e7ad7bca68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68630fd3d92240d0a6fa5353366799d2'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:22:35.415586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-b8baefa1-8167-4484-a017-c9e7ad7bca68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68630fd3d92240d0a6fa5353366799d2'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:22:35.416170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-b8baefa1-8167-4484-a017-c9e7ad7bca68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68630fd3d92240d0a6fa5353366799d2'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:22:35.432732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 749/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:22:35 2022] DELETE /identity/v3/users/68630fd3d92240d0a6fa5353366799d2 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:22:35.437327 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809969db-c94d-4e2f-96da-df8457838bb2 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:22:35.437656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809969db-c94d-4e2f-96da-df8457838bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:22:35.437966 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809969db-c94d-4e2f-96da-df8457838bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:22:35.498625 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-809969db-c94d-4e2f-96da-df8457838bb2 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:22:35.499849 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-809969db-c94d-4e2f-96da-df8457838bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:22:35.579201 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:22:35.592505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 759/1508] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:22:35.603241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baefa1-8167-4484-a017-c9e7ad7bca68 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:22:35.604529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b8baefa1-8167-4484-a017-c9e7ad7bca68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:22:35.710916 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8baefa1-8167-4484-a017-c9e7ad7bca68 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:22:35.769592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:22:35.790561 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3a284-a673-446e-b762-de8a29525f51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:22:35.791210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3a284-a673-446e-b762-de8a29525f51 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:22:35.791488 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3a284-a673-446e-b762-de8a29525f51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:22:35.791811 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3a284-a673-446e-b762-de8a29525f51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:22:35.793468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b1d3a284-a673-446e-b762-de8a29525f51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:22:35.909941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3a284-a673-446e-b762-de8a29525f51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:22:35.911361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3a284-a673-446e-b762-de8a29525f51 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:22:35.930940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 750/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:22:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:22:36.378112 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809969db-c94d-4e2f-96da-df8457838bb2 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:22:36.379363 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-809969db-c94d-4e2f-96da-df8457838bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:22:36.391365 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809969db-c94d-4e2f-96da-df8457838bb2 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:22:36.495221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416ebb3-cf90-46bb-815a-01daa3483347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:22:36.495796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416ebb3-cf90-46bb-815a-01daa3483347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:22:36.496097 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416ebb3-cf90-46bb-815a-01daa3483347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:22:36.496368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416ebb3-cf90-46bb-815a-01daa3483347 admin admin] PATH_INFO: `/v3/projects/c32036a7352f4e3da4548f522ccc5bca` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:22:36.506600 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416ebb3-cf90-46bb-815a-01daa3483347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c32036a7352f4e3da4548f522ccc5bca)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:22:36.507634 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: WARNING py.warnings [None req-0416ebb3-cf90-46bb-815a-01daa3483347 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 19:22:36.507634 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: warnings.warn(msg) Aug 07 19:22:36.507634 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: Aug 07 19:22:36.508901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416ebb3-cf90-46bb-815a-01daa3483347 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:22:36.570790 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0416ebb3-cf90-46bb-815a-01daa3483347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c32036a7352f4e3da4548f522ccc5bca', 'request_id': 'req-0416ebb3-cf90-46bb-815a-01daa3483347'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:22:36.576722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0416ebb3-cf90-46bb-815a-01daa3483347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c32036a7352f4e3da4548f522ccc5bca', 'request_id': 'req-0416ebb3-cf90-46bb-815a-01daa3483347'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:22:36.589801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0416ebb3-cf90-46bb-815a-01daa3483347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c32036a7352f4e3da4548f522ccc5bca', 'request_id': 'req-0416ebb3-cf90-46bb-815a-01daa3483347'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:22:36.618526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0416ebb3-cf90-46bb-815a-01daa3483347 admin admin] The token cache is being invalidate because project c32036a7352f4e3da4548f522ccc5bca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:22:36.618826 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0416ebb3-cf90-46bb-815a-01daa3483347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:22:36.619257 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0416ebb3-cf90-46bb-815a-01daa3483347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:22:36.620779 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 760/1510] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:22:36 2022] DELETE /identity/v3/projects/c32036a7352f4e3da4548f522ccc5bca => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:22:45.928235 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=62 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:22:46.616100 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:24:03.830765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507712a-4773-4501-9694-33120aa46d8c None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:03.831497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507712a-4773-4501-9694-33120aa46d8c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:03.831497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507712a-4773-4501-9694-33120aa46d8c None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:03.832740 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 19:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:24:03.839893 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408eedb-bd25-45ef-9ff9-855056f06c1f None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:03.840725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408eedb-bd25-45ef-9ff9-855056f06c1f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:03.840725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408eedb-bd25-45ef-9ff9-855056f06c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:03.966336 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-5408eedb-bd25-45ef-9ff9-855056f06c1f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:24:03.967849 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-5408eedb-bd25-45ef-9ff9-855056f06c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:24:04.098861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 19:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:24:04.271915 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507712a-4773-4501-9694-33120aa46d8c None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:24:04.275709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1507712a-4773-4501-9694-33120aa46d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:24:04.429105 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507712a-4773-4501-9694-33120aa46d8c None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:24:04.434922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa5e917-1660-4955-acba-3264db8adae2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:24:04.435555 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5e917-1660-4955-acba-3264db8adae2 service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:04.435841 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5e917-1660-4955-acba-3264db8adae2 service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:04.436148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa5e917-1660-4955-acba-3264db8adae2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:04.446208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5e917-1660-4955-acba-3264db8adae2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:24:04.448226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa5e917-1660-4955-acba-3264db8adae2 service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:24:04.460820 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 752/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:24:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:24:05.420916 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067f838-e081-4dc0-8158-d796f7e32df0 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:05.421785 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067f838-e081-4dc0-8158-d796f7e32df0 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:05.421967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067f838-e081-4dc0-8158-d796f7e32df0 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:05.423737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 19:24:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:24:05.430012 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a3e55-f865-49ab-a482-adbcd3c4e8b1 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:05.430541 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a3e55-f865-49ab-a482-adbcd3c4e8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:05.430653 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a3e55-f865-49ab-a482-adbcd3c4e8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:05.458271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-de1a3e55-f865-49ab-a482-adbcd3c4e8b1 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:24:05.459806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-de1a3e55-f865-49ab-a482-adbcd3c4e8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:24:05.469003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 19:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:24:05.495503 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f5118-a14a-4d15-9752-59193595a3b6 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:05.495885 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f5118-a14a-4d15-9752-59193595a3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:05.496274 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f5118-a14a-4d15-9752-59193595a3b6 None None] PATH_INFO: `/v3` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:05.497928 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 763/1516] 10.222.0.18 () {54 vars in 925 bytes} [Sun Aug 7 19:24:05 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:24:05.506353 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06115156-245c-4beb-b0ed-6c075b28fe6d None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:05.506782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06115156-245c-4beb-b0ed-6c075b28fe6d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:05.507079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06115156-245c-4beb-b0ed-6c075b28fe6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:05.580831 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-06115156-245c-4beb-b0ed-6c075b28fe6d None None] MFA Rules not processed for user `143c30b4201645279564e72ab3288adc`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:24:05.582169 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-06115156-245c-4beb-b0ed-6c075b28fe6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:24:05.641306 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f5118-a14a-4d15-9752-59193595a3b6 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:24:05.657942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:24:05.665325 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f5118-a14a-4d15-9752-59193595a3b6 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:24:05.676631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a676747-525c-46d7-b9d6-848a0f2bfd19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:24:05.677752 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a676747-525c-46d7-b9d6-848a0f2bfd19 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:05.678096 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a676747-525c-46d7-b9d6-848a0f2bfd19 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:05.678640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a676747-525c-46d7-b9d6-848a0f2bfd19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:05.678823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 754/1517] 10.222.0.18 () {58 vars in 1015 bytes} [Sun Aug 7 19:24:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:24:05.693753 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a676747-525c-46d7-b9d6-848a0f2bfd19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:24:05.695968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a676747-525c-46d7-b9d6-848a0f2bfd19 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:24:05.713329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 764/1518] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:24:14.324024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05385ad-f811-45e5-a3ae-df98ae3cc21b None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:14.324632 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05385ad-f811-45e5-a3ae-df98ae3cc21b None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:14.324632 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05385ad-f811-45e5-a3ae-df98ae3cc21b None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:14.325899 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 755/1519] 10.222.0.17 () {54 vars in 961 bytes} [Sun Aug 7 19:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:24:14.331560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829eb25-a0fd-4269-9cf3-a562bfd541b7 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:14.332048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829eb25-a0fd-4269-9cf3-a562bfd541b7 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:14.332153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829eb25-a0fd-4269-9cf3-a562bfd541b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:14.353525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-9829eb25-a0fd-4269-9cf3-a562bfd541b7 None None] MFA Rules not processed for user `143c30b4201645279564e72ab3288adc`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:24:14.354756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9829eb25-a0fd-4269-9cf3-a562bfd541b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:24:14.364370 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 765/1520] 10.222.0.17 () {58 vars in 1058 bytes} [Sun Aug 7 19:24:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:24:14.372447 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05385ad-f811-45e5-a3ae-df98ae3cc21b None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:24:14.386038 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05385ad-f811-45e5-a3ae-df98ae3cc21b None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:24:14.391105 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646be6c-a942-4dbe-9997-af01189b7778 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '143c30b4201645279564e72ab3288adc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:24:14.391821 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646be6c-a942-4dbe-9997-af01189b7778 service placement] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:14.392133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646be6c-a942-4dbe-9997-af01189b7778 service placement] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:14.392410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646be6c-a942-4dbe-9997-af01189b7778 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:14.400609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646be6c-a942-4dbe-9997-af01189b7778 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:24:14.402214 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646be6c-a942-4dbe-9997-af01189b7778 service placement] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:24:14.412317 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 756/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:24:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:24:16.993189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6b0dc-75ed-4470-a46a-24211d774396 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:16.993838 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6b0dc-75ed-4470-a46a-24211d774396 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:16.993838 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd6b0dc-75ed-4470-a46a-24211d774396 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:16.995093 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 19:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:24:17.000088 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fd98c-b23f-46bd-a8e6-ad850e66eef3 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:17.000525 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fd98c-b23f-46bd-a8e6-ad850e66eef3 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:17.000643 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fd98c-b23f-46bd-a8e6-ad850e66eef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:17.021450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-670fd98c-b23f-46bd-a8e6-ad850e66eef3 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:24:17.022670 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-670fd98c-b23f-46bd-a8e6-ad850e66eef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:24:17.031385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 19:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:24:17.186931 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6b0dc-75ed-4470-a46a-24211d774396 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:24:17.204399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd6b0dc-75ed-4470-a46a-24211d774396 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:24:17.211122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac06f1a-7ff7-4078-84d4-791bff8602b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:24:17.211756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac06f1a-7ff7-4078-84d4-791bff8602b8 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:17.212071 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac06f1a-7ff7-4078-84d4-791bff8602b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:17.212343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac06f1a-7ff7-4078-84d4-791bff8602b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:17.222390 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac06f1a-7ff7-4078-84d4-791bff8602b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:24:17.224493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac06f1a-7ff7-4078-84d4-791bff8602b8 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:24:17.235709 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 767/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:24:18.064274 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1826a3-cc3e-489d-a065-feecb0c7a807 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:18.064953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1826a3-cc3e-489d-a065-feecb0c7a807 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:18.064953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1826a3-cc3e-489d-a065-feecb0c7a807 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:18.066126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 19:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:24:18.072659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d800f32-8635-44c6-a6f9-efe66ecd93e5 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:18.073226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d800f32-8635-44c6-a6f9-efe66ecd93e5 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:18.073226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d800f32-8635-44c6-a6f9-efe66ecd93e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:18.094182 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-3d800f32-8635-44c6-a6f9-efe66ecd93e5 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:24:18.095475 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3d800f32-8635-44c6-a6f9-efe66ecd93e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:24:18.103651 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 19:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:24:18.256259 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1826a3-cc3e-489d-a065-feecb0c7a807 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:24:18.271650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1826a3-cc3e-489d-a065-feecb0c7a807 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:24:18.277768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2bb8eb-d5d9-49da-8fe1-a7879e50d196 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:24:18.278389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bb8eb-d5d9-49da-8fe1-a7879e50d196 service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:18.278691 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bb8eb-d5d9-49da-8fe1-a7879e50d196 service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:18.278963 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bb8eb-d5d9-49da-8fe1-a7879e50d196 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:18.288341 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2bb8eb-d5d9-49da-8fe1-a7879e50d196 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:24:18.289966 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2bb8eb-d5d9-49da-8fe1-a7879e50d196 service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:24:18.300780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 759/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:24:22.033403 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64859caf-b85c-4fe5-b445-b8ae2ab9e30d None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:22.034015 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64859caf-b85c-4fe5-b445-b8ae2ab9e30d None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:22.034015 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64859caf-b85c-4fe5-b445-b8ae2ab9e30d None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:22.035628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 19:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:24:22.041555 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1aef5f-94be-41ad-91aa-b94b2a8efd02 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:22.042075 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1aef5f-94be-41ad-91aa-b94b2a8efd02 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:22.042075 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1aef5f-94be-41ad-91aa-b94b2a8efd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:22.064309 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-8b1aef5f-94be-41ad-91aa-b94b2a8efd02 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:24:22.065745 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8b1aef5f-94be-41ad-91aa-b94b2a8efd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:24:22.075180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 19:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:24:22.162735 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64859caf-b85c-4fe5-b445-b8ae2ab9e30d None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:24:22.164610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-64859caf-b85c-4fe5-b445-b8ae2ab9e30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:24:22.199511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:24:22.249914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64859caf-b85c-4fe5-b445-b8ae2ab9e30d None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:24:22.254116 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ebfb99-d31e-42ec-a41e-14b49f9f5766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:24:22.254734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebfb99-d31e-42ec-a41e-14b49f9f5766 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:22.255011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebfb99-d31e-42ec-a41e-14b49f9f5766 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:22.255277 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ebfb99-d31e-42ec-a41e-14b49f9f5766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:22.265034 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ebfb99-d31e-42ec-a41e-14b49f9f5766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:24:22.266493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ebfb99-d31e-42ec-a41e-14b49f9f5766 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:24:22.276712 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 770/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:24:24.381449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6379e-c3ca-4471-a521-2717eb3a877c None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:24.382055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6379e-c3ca-4471-a521-2717eb3a877c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:24.382302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6379e-c3ca-4471-a521-2717eb3a877c None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:24.383683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 19:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:24:24.390433 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d3e84-0572-493b-b825-6f3aa2737a78 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:24.390973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d3e84-0572-493b-b825-6f3aa2737a78 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:24.390973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d3e84-0572-493b-b825-6f3aa2737a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:24.412221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-b52d3e84-0572-493b-b825-6f3aa2737a78 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:24:24.413457 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b52d3e84-0572-493b-b825-6f3aa2737a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:24:24.422641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 19:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:24:24.459066 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6379e-c3ca-4471-a521-2717eb3a877c None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:24:24.461104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b0c6379e-c3ca-4471-a521-2717eb3a877c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:24:24.476806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6379e-c3ca-4471-a521-2717eb3a877c None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:24:24.481055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30818025-a06e-4c22-bc82-bd5822178247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:24:24.481669 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30818025-a06e-4c22-bc82-bd5822178247 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:24.481959 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30818025-a06e-4c22-bc82-bd5822178247 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:24.482229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30818025-a06e-4c22-bc82-bd5822178247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:24.492278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30818025-a06e-4c22-bc82-bd5822178247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:24:24.493753 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30818025-a06e-4c22-bc82-bd5822178247 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:24:24.504492 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 762/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:24:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:24:27.977511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e2042-0152-4675-a3e8-f0efa7740045 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:27.978091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e2042-0152-4675-a3e8-f0efa7740045 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:27.978091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e2042-0152-4675-a3e8-f0efa7740045 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:27.979473 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 19:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:24:27.984756 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada5db-bd1b-4c05-8287-f4c645a61bbe None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:27.985242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada5db-bd1b-4c05-8287-f4c645a61bbe None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:27.985377 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada5db-bd1b-4c05-8287-f4c645a61bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:28.005322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-80ada5db-bd1b-4c05-8287-f4c645a61bbe None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:24:28.006531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-80ada5db-bd1b-4c05-8287-f4c645a61bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:24:28.015217 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 19:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:24:28.319760 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e2042-0152-4675-a3e8-f0efa7740045 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:24:28.321669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-029e2042-0152-4675-a3e8-f0efa7740045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:24:28.405111 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e2042-0152-4675-a3e8-f0efa7740045 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:24:28.410141 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196603d0-8982-4ddb-80a5-225c0f700b07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:24:28.410763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196603d0-8982-4ddb-80a5-225c0f700b07 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:28.411033 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196603d0-8982-4ddb-80a5-225c0f700b07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:28.411300 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196603d0-8982-4ddb-80a5-225c0f700b07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:28.421345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196603d0-8982-4ddb-80a5-225c0f700b07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:24:28.422784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196603d0-8982-4ddb-80a5-225c0f700b07 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:24:28.432849 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 773/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:24:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:24:30.465676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa0bb1-eb86-4c2e-a777-421f2b39e55a None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:30.466264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa0bb1-eb86-4c2e-a777-421f2b39e55a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:30.466264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa0bb1-eb86-4c2e-a777-421f2b39e55a None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:30.467623 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 19:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:24:30.474014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42da2b-476a-4e0d-b528-5d2380fb36a7 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:30.474566 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42da2b-476a-4e0d-b528-5d2380fb36a7 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:30.474837 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42da2b-476a-4e0d-b528-5d2380fb36a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:30.502127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-5e42da2b-476a-4e0d-b528-5d2380fb36a7 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:24:30.503447 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-5e42da2b-476a-4e0d-b528-5d2380fb36a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:24:30.512138 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 19:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:24:30.823493 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa0bb1-eb86-4c2e-a777-421f2b39e55a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:24:30.825339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-73aa0bb1-eb86-4c2e-a777-421f2b39e55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:24:30.839510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa0bb1-eb86-4c2e-a777-421f2b39e55a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:24:30.843334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a9073-6ac4-478b-914f-a15c46ccbc49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:24:30.843950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a9073-6ac4-478b-914f-a15c46ccbc49 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:24:30.844244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a9073-6ac4-478b-914f-a15c46ccbc49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:24:30.844548 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a9073-6ac4-478b-914f-a15c46ccbc49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:24:30.854555 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3a9073-6ac4-478b-914f-a15c46ccbc49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:24:30.855972 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3a9073-6ac4-478b-914f-a15c46ccbc49 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:24:30.866529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 765/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:24:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:24:40.493761 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=63 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:24:40.861574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=37 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:51.909497 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a463b-b9c9-44ce-97c3-c9bbb0b8d24f None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:51.926872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a463b-b9c9-44ce-97c3-c9bbb0b8d24f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:51.926872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a463b-b9c9-44ce-97c3-c9bbb0b8d24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:51.939023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-d86a463b-b9c9-44ce-97c3-c9bbb0b8d24f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:51.940706 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-d86a463b-b9c9-44ce-97c3-c9bbb0b8d24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:51.954847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 775/1540] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:51.964595 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296f290-053d-4309-9365-0370d9846b0c None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:51.965220 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296f290-053d-4309-9365-0370d9846b0c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:51.965220 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8296f290-053d-4309-9365-0370d9846b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:51.995022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-8296f290-053d-4309-9365-0370d9846b0c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:51.996273 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8296f290-053d-4309-9365-0370d9846b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:52.005166 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 766/1541] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:25:52.010221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a463b-b9c9-44ce-97c3-c9bbb0b8d24f None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:52.010677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce798df-1540-47e3-a79f-aa2ffdd85caa None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:52.010965 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce798df-1540-47e3-a79f-aa2ffdd85caa None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:52.011248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce798df-1540-47e3-a79f-aa2ffdd85caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:52.038595 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a463b-b9c9-44ce-97c3-c9bbb0b8d24f None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:52.044773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37ffbd-a2be-4b5a-a968-db9257f3b7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:52.045858 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37ffbd-a2be-4b5a-a968-db9257f3b7cd admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:52.052189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37ffbd-a2be-4b5a-a968-db9257f3b7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:52.052189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37ffbd-a2be-4b5a-a968-db9257f3b7cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:52.052189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37ffbd-a2be-4b5a-a968-db9257f3b7cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:25:52.052189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37ffbd-a2be-4b5a-a968-db9257f3b7cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:52.052801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-fce798df-1540-47e3-a79f-aa2ffdd85caa None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:52.054122 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-fce798df-1540-47e3-a79f-aa2ffdd85caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:52.055553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37ffbd-a2be-4b5a-a968-db9257f3b7cd admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:52.067278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 767/1542] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:25:52.084793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22693ca5-b100-4860-95eb-3cb426e7f774 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:52.085151 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22693ca5-b100-4860-95eb-3cb426e7f774 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:52.085380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22693ca5-b100-4860-95eb-3cb426e7f774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:52.088436 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 776/1543] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:25:52.117654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37ffbd-a2be-4b5a-a968-db9257f3b7cd admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:52.123400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-22693ca5-b100-4860-95eb-3cb426e7f774 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:52.124685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-22693ca5-b100-4860-95eb-3cb426e7f774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:52.134921 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37ffbd-a2be-4b5a-a968-db9257f3b7cd admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:52.140631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64744f6-2108-4c7a-b73b-27591a671149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:52.141660 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64744f6-2108-4c7a-b73b-27591a671149 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:52.141792 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 768/1544] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:52.142289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64744f6-2108-4c7a-b73b-27591a671149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:52.142974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64744f6-2108-4c7a-b73b-27591a671149 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:52.143835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64744f6-2108-4c7a-b73b-27591a671149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:52.145166 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: WARNING py.warnings [None req-f64744f6-2108-4c7a-b73b-27591a671149 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 19:25:52.145166 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: warnings.warn(msg) Aug 07 19:25:52.145166 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: Aug 07 19:25:52.146777 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64744f6-2108-4c7a-b73b-27591a671149 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:52.153139 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22693ca5-b100-4860-95eb-3cb426e7f774 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:52.167496 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22693ca5-b100-4860-95eb-3cb426e7f774 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:52.185120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f322f0-ccb2-4baa-b739-163cb36233b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:52.185765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f322f0-ccb2-4baa-b739-163cb36233b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:52.186043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f322f0-ccb2-4baa-b739-163cb36233b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:52.186314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f322f0-ccb2-4baa-b739-163cb36233b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:52.186982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f322f0-ccb2-4baa-b739-163cb36233b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:25:52.187288 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f322f0-ccb2-4baa-b739-163cb36233b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:52.188348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: WARNING py.warnings [None req-c1f322f0-ccb2-4baa-b739-163cb36233b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 19:25:52.188348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: warnings.warn(msg) Aug 07 19:25:52.188348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: Aug 07 19:25:52.189247 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f322f0-ccb2-4baa-b739-163cb36233b8 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:52.207897 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 769/1545] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:25:52.221226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f322f0-ccb2-4baa-b739-163cb36233b8 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:52.225595 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 777/1546] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:25:52 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:52.233415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64744f6-2108-4c7a-b73b-27591a671149 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:52.237969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f322f0-ccb2-4baa-b739-163cb36233b8 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:52.360764 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd535f-becc-4bae-95fc-bcbbf28cc126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:52.361446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd535f-becc-4bae-95fc-bcbbf28cc126 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:52.361730 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd535f-becc-4bae-95fc-bcbbf28cc126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:52.362006 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd535f-becc-4bae-95fc-bcbbf28cc126 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:52.362671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd535f-becc-4bae-95fc-bcbbf28cc126 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:52.364705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd535f-becc-4bae-95fc-bcbbf28cc126 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:52.380070 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64744f6-2108-4c7a-b73b-27591a671149 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:52.385951 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ef969-a2a1-43c4-a922-5d45704405cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:52.386942 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ef969-a2a1-43c4-a922-5d45704405cf admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:52.387448 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ef969-a2a1-43c4-a922-5d45704405cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:52.391789 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ef969-a2a1-43c4-a922-5d45704405cf admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:52.393554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ef969-a2a1-43c4-a922-5d45704405cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:52.396350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ef969-a2a1-43c4-a922-5d45704405cf admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:52.426663 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 770/1547] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:25:52 2022] POST /identity/v3/projects => generated 379 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:25:52.429339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca816a8-f9d8-4cce-b462-0807bc475c57 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:52.429678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca816a8-f9d8-4cce-b462-0807bc475c57 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:52.429891 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca816a8-f9d8-4cce-b462-0807bc475c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:52.466346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-1ca816a8-f9d8-4cce-b462-0807bc475c57 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:52.468007 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1ca816a8-f9d8-4cce-b462-0807bc475c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:52.476023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 778/1548] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:25:52 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:25:52.476023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ef969-a2a1-43c4-a922-5d45704405cf admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:52.498703 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ef969-a2a1-43c4-a922-5d45704405cf admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:52.616876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac1181-a8a0-4dfe-9a39-273badd30f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:52.617538 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac1181-a8a0-4dfe-9a39-273badd30f23 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:52.617813 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac1181-a8a0-4dfe-9a39-273badd30f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:52.618080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac1181-a8a0-4dfe-9a39-273badd30f23 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:52.618727 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac1181-a8a0-4dfe-9a39-273badd30f23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:52.620339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 771/1549] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:52.620917 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ac1181-a8a0-4dfe-9a39-273badd30f23 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:52.621878 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca816a8-f9d8-4cce-b462-0807bc475c57 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:52.651769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca816a8-f9d8-4cce-b462-0807bc475c57 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:52.666879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866f551-a919-4f90-aff7-3cf8345e1bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:52.667174 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 779/1550] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:25:52 2022] POST /identity/v3/users => generated 290 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:52.668136 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866f551-a919-4f90-aff7-3cf8345e1bcc admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:52.668375 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24186e-52f5-46b7-82fd-8116e9e401ef None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:52.668622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866f551-a919-4f90-aff7-3cf8345e1bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:52.668727 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24186e-52f5-46b7-82fd-8116e9e401ef None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:52.668927 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24186e-52f5-46b7-82fd-8116e9e401ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:52.669115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866f551-a919-4f90-aff7-3cf8345e1bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:52.673946 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866f551-a919-4f90-aff7-3cf8345e1bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:52.674142 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866f551-a919-4f90-aff7-3cf8345e1bcc admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:52.721360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-0d24186e-52f5-46b7-82fd-8116e9e401ef None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:52.722680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0d24186e-52f5-46b7-82fd-8116e9e401ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:52.723268 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 772/1551] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:52.724754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866f551-a919-4f90-aff7-3cf8345e1bcc admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:52.756727 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 780/1552] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:52.757912 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24186e-52f5-46b7-82fd-8116e9e401ef None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:52.768543 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866f551-a919-4f90-aff7-3cf8345e1bcc admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:52.784307 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d24186e-52f5-46b7-82fd-8116e9e401ef None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:52.787466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50950f9d-e60d-4f65-9b3f-747de7b08431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:52.788592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50950f9d-e60d-4f65-9b3f-747de7b08431 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:52.789080 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50950f9d-e60d-4f65-9b3f-747de7b08431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:52.789531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50950f9d-e60d-4f65-9b3f-747de7b08431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:52.790724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50950f9d-e60d-4f65-9b3f-747de7b08431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:52.793958 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50950f9d-e60d-4f65-9b3f-747de7b08431 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:52.809596 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c388a-ab67-4998-91e0-d5194c9f50b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:52.810241 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c388a-ab67-4998-91e0-d5194c9f50b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:52.810512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c388a-ab67-4998-91e0-d5194c9f50b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:52.810932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c388a-ab67-4998-91e0-d5194c9f50b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:52.811593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c388a-ab67-4998-91e0-d5194c9f50b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:52.813271 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c388a-ab67-4998-91e0-d5194c9f50b8 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:52.830055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 773/1553] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:52.830398 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 781/1554] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:52.831548 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50950f9d-e60d-4f65-9b3f-747de7b08431 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:52.837332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c388a-ab67-4998-91e0-d5194c9f50b8 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:52.856590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c388a-ab67-4998-91e0-d5194c9f50b8 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:52.867086 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50950f9d-e60d-4f65-9b3f-747de7b08431 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:52.869159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19798b74-de70-4f22-8500-42cf337edf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:52.869862 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19798b74-de70-4f22-8500-42cf337edf9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:52.870133 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19798b74-de70-4f22-8500-42cf337edf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:52.870402 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19798b74-de70-4f22-8500-42cf337edf9e admin admin] PATH_INFO: `/v3/projects/b10e8224d7aa4b019ccad26aa3398933/users/be3e2afc428a45e2b993c39e3d53cd8b/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:52.882880 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b47528-6574-4b41-b640-2206bb0fbe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:52.883610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:52.884685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b47528-6574-4b41-b640-2206bb0fbe0a admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:52.884807 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b47528-6574-4b41-b640-2206bb0fbe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:52.885257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b47528-6574-4b41-b640-2206bb0fbe0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:52.886615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b47528-6574-4b41-b640-2206bb0fbe0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:25:52.887023 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b47528-6574-4b41-b640-2206bb0fbe0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:52.890013 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b47528-6574-4b41-b640-2206bb0fbe0a admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:52.897502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19798b74-de70-4f22-8500-42cf337edf9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b10e8224d7aa4b019ccad26aa3398933, user_id=be3e2afc428a45e2b993c39e3d53cd8b, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:52.898714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: WARNING py.warnings [None req-19798b74-de70-4f22-8500-42cf337edf9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 19:25:52.898714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: warnings.warn(msg) Aug 07 19:25:52.898714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: Aug 07 19:25:52.900726 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19798b74-de70-4f22-8500-42cf337edf9e admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:52.930784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 782/1555] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:25:52 2022] PUT /identity/v3/projects/b10e8224d7aa4b019ccad26aa3398933/users/be3e2afc428a45e2b993c39e3d53cd8b/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:25:52.931681 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19798b74-de70-4f22-8500-42cf337edf9e admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:52.936149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 774/1556] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:25:52.940127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41736b20-5bad-42f7-b378-2974feae7edc None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:52.940607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41736b20-5bad-42f7-b378-2974feae7edc None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:52.941084 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41736b20-5bad-42f7-b378-2974feae7edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.035615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-41736b20-5bad-42f7-b378-2974feae7edc None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:53.037537 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19798b74-de70-4f22-8500-42cf337edf9e admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:53.037788 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-41736b20-5bad-42f7-b378-2974feae7edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:53.059495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 775/1557] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:25:53.060921 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41736b20-5bad-42f7-b378-2974feae7edc None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:53.071967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59446fa8-1393-4525-a583-083c4ee0c114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:53.077027 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59446fa8-1393-4525-a583-083c4ee0c114 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.077330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59446fa8-1393-4525-a583-083c4ee0c114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.077581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59446fa8-1393-4525-a583-083c4ee0c114 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.078407 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59446fa8-1393-4525-a583-083c4ee0c114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:53.080173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59446fa8-1393-4525-a583-083c4ee0c114 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:53.091004 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41736b20-5bad-42f7-b378-2974feae7edc None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:53.104867 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 783/1558] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:53.106153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7b4d0-65fd-4bfc-ac8a-3fb468e2e58c None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.106443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7b4d0-65fd-4bfc-ac8a-3fb468e2e58c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.106718 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7b4d0-65fd-4bfc-ac8a-3fb468e2e58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.106950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639568d9-e942-46a2-b2ac-4db9cd762ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:53.108018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639568d9-e942-46a2-b2ac-4db9cd762ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.108563 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639568d9-e942-46a2-b2ac-4db9cd762ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.109024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639568d9-e942-46a2-b2ac-4db9cd762ad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.110228 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639568d9-e942-46a2-b2ac-4db9cd762ad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:53.113907 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639568d9-e942-46a2-b2ac-4db9cd762ad6 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:53.136992 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-cea7b4d0-65fd-4bfc-ac8a-3fb468e2e58c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:53.138243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-cea7b4d0-65fd-4bfc-ac8a-3fb468e2e58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:53.144423 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:53.162338 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 784/1559] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:53.163062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7b4d0-65fd-4bfc-ac8a-3fb468e2e58c None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:53.192748 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 776/1560] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:25:53 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:53.194022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639568d9-e942-46a2-b2ac-4db9cd762ad6 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:53.289575 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7b4d0-65fd-4bfc-ac8a-3fb468e2e58c None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:53.357675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d0628-5b6b-42e7-a9b5-4c7437505ce6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:53.358313 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d0628-5b6b-42e7-a9b5-4c7437505ce6 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.358591 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d0628-5b6b-42e7-a9b5-4c7437505ce6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.358855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d0628-5b6b-42e7-a9b5-4c7437505ce6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.384396 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d0628-5b6b-42e7-a9b5-4c7437505ce6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:53.385868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d0628-5b6b-42e7-a9b5-4c7437505ce6 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:53.390231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639568d9-e942-46a2-b2ac-4db9cd762ad6 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:53.407661 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71daa3c-3340-4e8c-88f8-7dc65460cdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:53.409141 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71daa3c-3340-4e8c-88f8-7dc65460cdfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.409613 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71daa3c-3340-4e8c-88f8-7dc65460cdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.412329 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71daa3c-3340-4e8c-88f8-7dc65460cdfe admin admin] PATH_INFO: `/v3/projects/d300969a86954ad7ab74b743e72bcaf1/users/5ce40a75512a4ff9971552ffc54136c3/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.414547 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 785/1561] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:25:53.419773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629d9aa-aff5-423f-a9cc-bc8557f9cb07 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.422777 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629d9aa-aff5-423f-a9cc-bc8557f9cb07 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.423641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629d9aa-aff5-423f-a9cc-bc8557f9cb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.461846 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71daa3c-3340-4e8c-88f8-7dc65460cdfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d300969a86954ad7ab74b743e72bcaf1, user_id=5ce40a75512a4ff9971552ffc54136c3, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:53.466207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71daa3c-3340-4e8c-88f8-7dc65460cdfe admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:53.478223 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-9629d9aa-aff5-423f-a9cc-bc8557f9cb07 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:53.479922 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9629d9aa-aff5-423f-a9cc-bc8557f9cb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:53.500592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 777/1562] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:25:53 2022] PUT /identity/v3/projects/d300969a86954ad7ab74b743e72bcaf1/users/5ce40a75512a4ff9971552ffc54136c3/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:25:53.503111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71daa3c-3340-4e8c-88f8-7dc65460cdfe admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:53.524566 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:53.571061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 786/1563] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:53.572928 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42285137-338b-49e9-94c6-aa7013dc1425 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.573338 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42285137-338b-49e9-94c6-aa7013dc1425 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.573855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42285137-338b-49e9-94c6-aa7013dc1425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.598869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-42285137-338b-49e9-94c6-aa7013dc1425 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:53.600373 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-42285137-338b-49e9-94c6-aa7013dc1425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:53.614420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 787/1564] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:53.624927 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af391de0-7d33-4082-a28c-7f58cbe408d2 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.625422 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af391de0-7d33-4082-a28c-7f58cbe408d2 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.625890 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af391de0-7d33-4082-a28c-7f58cbe408d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.662805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-af391de0-7d33-4082-a28c-7f58cbe408d2 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:53.664113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-af391de0-7d33-4082-a28c-7f58cbe408d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:53.668911 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71daa3c-3340-4e8c-88f8-7dc65460cdfe admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:53.680084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 788/1565] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:53.680084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ed2ab-56e4-4b06-bbd6-dcf009dd48cb None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.680084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ed2ab-56e4-4b06-bbd6-dcf009dd48cb None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.680084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ed2ab-56e4-4b06-bbd6-dcf009dd48cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.680580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fc898-bd10-4df2-88e6-f8757ffda4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:53.680580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fc898-bd10-4df2-88e6-f8757ffda4de admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.680580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fc898-bd10-4df2-88e6-f8757ffda4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.680580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fc898-bd10-4df2-88e6-f8757ffda4de admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.680938 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93fc898-bd10-4df2-88e6-f8757ffda4de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:53.680938 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: WARNING py.warnings [None req-e93fc898-bd10-4df2-88e6-f8757ffda4de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 19:25:53.680938 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: warnings.warn(msg) Aug 07 19:25:53.680938 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: Aug 07 19:25:53.682331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93fc898-bd10-4df2-88e6-f8757ffda4de admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:53.700876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-028ed2ab-56e4-4b06-bbd6-dcf009dd48cb None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:53.702385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-028ed2ab-56e4-4b06-bbd6-dcf009dd48cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:53.724090 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 789/1566] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:53.727171 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126df175-182b-4876-bbe6-965f48669285 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.727725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126df175-182b-4876-bbe6-965f48669285 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.728525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126df175-182b-4876-bbe6-965f48669285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.733163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 778/1567] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:25:53 2022] POST /identity/v3/users => generated 288 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:53.734062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fc898-bd10-4df2-88e6-f8757ffda4de admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:53.756224 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fc898-bd10-4df2-88e6-f8757ffda4de admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:53.761018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-126df175-182b-4876-bbe6-965f48669285 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:53.762441 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-126df175-182b-4876-bbe6-965f48669285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:53.768924 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cad496-39e1-4fbc-88a0-f9111735b03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:53.770087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cad496-39e1-4fbc-88a0-f9111735b03e admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.771053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cad496-39e1-4fbc-88a0-f9111735b03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.771764 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cad496-39e1-4fbc-88a0-f9111735b03e admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.772991 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cad496-39e1-4fbc-88a0-f9111735b03e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:25:53.773520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cad496-39e1-4fbc-88a0-f9111735b03e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:53.773773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 790/1568] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:53.775806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3b7bd-42eb-4d49-9e0e-913b09e0d3f7 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.776240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cad496-39e1-4fbc-88a0-f9111735b03e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:53.778415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3b7bd-42eb-4d49-9e0e-913b09e0d3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.778415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3b7bd-42eb-4d49-9e0e-913b09e0d3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.811336 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 779/1569] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:25:53.813129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69438762-cd56-4142-8dfe-fcf248ee2426 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.813472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69438762-cd56-4142-8dfe-fcf248ee2426 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.813691 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69438762-cd56-4142-8dfe-fcf248ee2426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.817737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-aaf3b7bd-42eb-4d49-9e0e-913b09e0d3f7 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:53.819306 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-aaf3b7bd-42eb-4d49-9e0e-913b09e0d3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:53.829818 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 791/1570] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:53.832371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864cc83-c5e6-4223-86f0-8c2d10c5b855 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.832928 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864cc83-c5e6-4223-86f0-8c2d10c5b855 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.833434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864cc83-c5e6-4223-86f0-8c2d10c5b855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.852466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-69438762-cd56-4142-8dfe-fcf248ee2426 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:53.856163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-69438762-cd56-4142-8dfe-fcf248ee2426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:53.867079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-3864cc83-c5e6-4223-86f0-8c2d10c5b855 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:53.868783 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3864cc83-c5e6-4223-86f0-8c2d10c5b855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:53.872086 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 780/1571] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:53.873673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860640c-2c82-435e-b025-ce169d4daec2 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.874229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860640c-2c82-435e-b025-ce169d4daec2 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.874589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860640c-2c82-435e-b025-ce169d4daec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.880064 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 792/1572] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:53.881426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864cc83-c5e6-4223-86f0-8c2d10c5b855 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:53.894272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3864cc83-c5e6-4223-86f0-8c2d10c5b855 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:53.900741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3bc99-4353-408b-a574-346dce051b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:53.901381 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3bc99-4353-408b-a574-346dce051b25 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.901667 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3bc99-4353-408b-a574-346dce051b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.901936 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3bc99-4353-408b-a574-346dce051b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.902865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a3bc99-4353-408b-a574-346dce051b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:53.905322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a3bc99-4353-408b-a574-346dce051b25 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:53.906097 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-1860640c-2c82-435e-b025-ce169d4daec2 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:53.907879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1860640c-2c82-435e-b025-ce169d4daec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:53.916740 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 781/1573] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:53.917428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860640c-2c82-435e-b025-ce169d4daec2 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:53.931655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 793/1574] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:53.932332 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860640c-2c82-435e-b025-ce169d4daec2 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:53.932438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3bc99-4353-408b-a574-346dce051b25 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:53.941137 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af58a5-ae7d-4b5e-b3fd-b9eb892954b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:53.942410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af58a5-ae7d-4b5e-b3fd-b9eb892954b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:53.944338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af58a5-ae7d-4b5e-b3fd-b9eb892954b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:53.944697 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af58a5-ae7d-4b5e-b3fd-b9eb892954b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:53.945968 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af58a5-ae7d-4b5e-b3fd-b9eb892954b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:53.949858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0af58a5-ae7d-4b5e-b3fd-b9eb892954b2 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:53.990707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 782/1575] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:25:53 2022] POST /identity/v3/projects => generated 363 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:53.991668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af58a5-ae7d-4b5e-b3fd-b9eb892954b2 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:54.015021 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:54.032125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3bc99-4353-408b-a574-346dce051b25 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:54.088012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369e8a5-498d-4b90-bee4-af0a5a28126b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:54.089186 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369e8a5-498d-4b90-bee4-af0a5a28126b service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:54.089186 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369e8a5-498d-4b90-bee4-af0a5a28126b service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:54.089403 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af58a5-ae7d-4b5e-b3fd-b9eb892954b2 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:54.089864 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369e8a5-498d-4b90-bee4-af0a5a28126b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:54.097068 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db09d6d8-ecab-4673-ae20-f6add4f82959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:54.098439 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09d6d8-ecab-4673-ae20-f6add4f82959 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:54.098855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09d6d8-ecab-4673-ae20-f6add4f82959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:54.099203 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db09d6d8-ecab-4673-ae20-f6add4f82959 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:54.099978 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09d6d8-ecab-4673-ae20-f6add4f82959 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:25:54.100184 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2369e8a5-498d-4b90-bee4-af0a5a28126b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:54.100473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09d6d8-ecab-4673-ae20-f6add4f82959 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:54.101566 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2369e8a5-498d-4b90-bee4-af0a5a28126b service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:54.102122 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db09d6d8-ecab-4673-ae20-f6add4f82959 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:54.112744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 794/1576] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:25:54.113717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369e8a5-498d-4b90-bee4-af0a5a28126b service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:54.116235 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 783/1577] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:25:54.118505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9e37f-d72f-41f2-976f-613beb3e17b9 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:54.118783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9e37f-d72f-41f2-976f-613beb3e17b9 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:54.119066 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9e37f-d72f-41f2-976f-613beb3e17b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:54.125456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2369e8a5-498d-4b90-bee4-af0a5a28126b service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:54.132756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fb53a-e3a8-4379-a8bc-d54520cc9dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:54.133438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fb53a-e3a8-4379-a8bc-d54520cc9dd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:54.133722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fb53a-e3a8-4379-a8bc-d54520cc9dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:54.134003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44fb53a-e3a8-4379-a8bc-d54520cc9dd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:54.134809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fb53a-e3a8-4379-a8bc-d54520cc9dd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:25:54.135114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fb53a-e3a8-4379-a8bc-d54520cc9dd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:54.136944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44fb53a-e3a8-4379-a8bc-d54520cc9dd6 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:54.163310 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 795/1578] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:25:54.163310 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fb53a-e3a8-4379-a8bc-d54520cc9dd6 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:54.163603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-e0f9e37f-d72f-41f2-976f-613beb3e17b9 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:54.163603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e0f9e37f-d72f-41f2-976f-613beb3e17b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:54.167809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44fb53a-e3a8-4379-a8bc-d54520cc9dd6 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:54.170013 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 784/1579] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:54.170998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9e37f-d72f-41f2-976f-613beb3e17b9 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:54.177948 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5787f-ecf3-4d92-8301-97413fcd4d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:54.179101 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5787f-ecf3-4d92-8301-97413fcd4d50 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:54.179445 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5787f-ecf3-4d92-8301-97413fcd4d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:54.179804 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5787f-ecf3-4d92-8301-97413fcd4d50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:54.181289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5787f-ecf3-4d92-8301-97413fcd4d50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:25:54.181669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5787f-ecf3-4d92-8301-97413fcd4d50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:54.183488 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5787f-ecf3-4d92-8301-97413fcd4d50 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:54.184643 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9e37f-d72f-41f2-976f-613beb3e17b9 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:54.190434 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602c4cb-2169-4d48-a7c1-66abcff28fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:54.191418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602c4cb-2169-4d48-a7c1-66abcff28fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:54.191898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602c4cb-2169-4d48-a7c1-66abcff28fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:54.192574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602c4cb-2169-4d48-a7c1-66abcff28fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:54.193505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602c4cb-2169-4d48-a7c1-66abcff28fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:54.195360 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602c4cb-2169-4d48-a7c1-66abcff28fc1 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:54.208703 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 796/1580] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:25:54.208703 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5787f-ecf3-4d92-8301-97413fcd4d50 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:54.216162 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 785/1581] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:54.217559 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602c4cb-2169-4d48-a7c1-66abcff28fc1 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:54.226530 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5787f-ecf3-4d92-8301-97413fcd4d50 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:54.237003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602c4cb-2169-4d48-a7c1-66abcff28fc1 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:54.237445 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197d80a-afb6-4fe7-9925-667d2eced9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:54.238991 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197d80a-afb6-4fe7-9925-667d2eced9d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:54.239268 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197d80a-afb6-4fe7-9925-667d2eced9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:54.239927 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197d80a-afb6-4fe7-9925-667d2eced9d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:54.241454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197d80a-afb6-4fe7-9925-667d2eced9d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:54.243885 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197d80a-afb6-4fe7-9925-667d2eced9d7 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:54.252462 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4f77a-56bc-43cd-aae7-67f1ea76e19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:54.254130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4f77a-56bc-43cd-aae7-67f1ea76e19a admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:54.254484 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4f77a-56bc-43cd-aae7-67f1ea76e19a admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:54.254790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4f77a-56bc-43cd-aae7-67f1ea76e19a admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:54.255556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b4f77a-56bc-43cd-aae7-67f1ea76e19a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:54.259743 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b4f77a-56bc-43cd-aae7-67f1ea76e19a admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:54.271371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:54.284824 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 797/1582] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:25:54 2022] POST /identity/v3/users => generated 282 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:54.286784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197d80a-afb6-4fe7-9925-667d2eced9d7 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:54.300665 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197d80a-afb6-4fe7-9925-667d2eced9d7 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:54.305675 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 786/1583] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:25:54 2022] POST /identity/v3/projects => generated 381 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:54.305675 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4f77a-56bc-43cd-aae7-67f1ea76e19a admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:54.436020 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4f77a-56bc-43cd-aae7-67f1ea76e19a admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:54.436231 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538a927-6773-4556-b4ee-04d39911e675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:54.436787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538a927-6773-4556-b4ee-04d39911e675 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:54.437104 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538a927-6773-4556-b4ee-04d39911e675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:54.437287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538a927-6773-4556-b4ee-04d39911e675 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:54.437907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538a927-6773-4556-b4ee-04d39911e675 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:54.440009 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538a927-6773-4556-b4ee-04d39911e675 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:54.442410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3348242-dfaf-408d-ab1b-af55a616db0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:54.443076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348242-dfaf-408d-ab1b-af55a616db0d admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:54.443360 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348242-dfaf-408d-ab1b-af55a616db0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:54.443751 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3348242-dfaf-408d-ab1b-af55a616db0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:54.444540 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3348242-dfaf-408d-ab1b-af55a616db0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:25:54.444978 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3348242-dfaf-408d-ab1b-af55a616db0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:54.447401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3348242-dfaf-408d-ab1b-af55a616db0d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:54.467182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 787/1584] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:25:54.468776 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3348242-dfaf-408d-ab1b-af55a616db0d admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:54.485455 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3348242-dfaf-408d-ab1b-af55a616db0d admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:54.494695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 798/1585] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:25:54 2022] POST /identity/v3/projects => generated 393 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:54.496151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538a927-6773-4556-b4ee-04d39911e675 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:54.500828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cc5eb-e27a-4c94-bbe9-acadce3c28cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:54.502176 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cc5eb-e27a-4c94-bbe9-acadce3c28cf admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:54.504063 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cc5eb-e27a-4c94-bbe9-acadce3c28cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:54.504814 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878cc5eb-e27a-4c94-bbe9-acadce3c28cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:54.505975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cc5eb-e27a-4c94-bbe9-acadce3c28cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:54.510134 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878cc5eb-e27a-4c94-bbe9-acadce3c28cf admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:54.564200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 788/1586] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:25:54 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:54.565851 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cc5eb-e27a-4c94-bbe9-acadce3c28cf admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:54.583656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538a927-6773-4556-b4ee-04d39911e675 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:54.632732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:54.663358 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878cc5eb-e27a-4c94-bbe9-acadce3c28cf admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:54.671904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ccb4f1-2972-4d1b-9326-3de2d3ab4bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:54.672441 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccb4f1-2972-4d1b-9326-3de2d3ab4bae admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:54.672750 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccb4f1-2972-4d1b-9326-3de2d3ab4bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:54.673455 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccb4f1-2972-4d1b-9326-3de2d3ab4bae admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:54.674710 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ccb4f1-2972-4d1b-9326-3de2d3ab4bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:54.681090 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ccb4f1-2972-4d1b-9326-3de2d3ab4bae admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:54.690651 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ef014-06b9-48ed-8268-d5159b38a04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:54.692943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ef014-06b9-48ed-8268-d5159b38a04a admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:54.696083 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ef014-06b9-48ed-8268-d5159b38a04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:54.696251 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 789/1587] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:54.696353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ef014-06b9-48ed-8268-d5159b38a04a admin admin] PATH_INFO: `/v3/projects/fcf93b388fac4be38661d185a7231937/users/6732487041c54918b1164b1cb72bb07e/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:54.696454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ccb4f1-2972-4d1b-9326-3de2d3ab4bae admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:54.708310 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ccb4f1-2972-4d1b-9326-3de2d3ab4bae admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:54.714921 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776da112-fd17-44e8-8103-687d311b8377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:54.716037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776da112-fd17-44e8-8103-687d311b8377 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:54.716037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776da112-fd17-44e8-8103-687d311b8377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:54.716257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776da112-fd17-44e8-8103-687d311b8377 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:54.717145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776da112-fd17-44e8-8103-687d311b8377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:54.721324 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776da112-fd17-44e8-8103-687d311b8377 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:54.723115 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ef014-06b9-48ed-8268-d5159b38a04a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcf93b388fac4be38661d185a7231937, user_id=6732487041c54918b1164b1cb72bb07e, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:54.729013 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ef014-06b9-48ed-8268-d5159b38a04a admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:54.750181 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 790/1588] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:25:54 2022] POST /identity/v3/users => generated 291 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:54.751628 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776da112-fd17-44e8-8103-687d311b8377 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:54.754986 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 799/1589] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:25:54 2022] PUT /identity/v3/projects/fcf93b388fac4be38661d185a7231937/users/6732487041c54918b1164b1cb72bb07e/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:25:54.756316 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ef014-06b9-48ed-8268-d5159b38a04a admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:54.764918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776da112-fd17-44e8-8103-687d311b8377 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:54.796239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:54.864471 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ef014-06b9-48ed-8268-d5159b38a04a admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:54.870454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4dad3-49f8-4dc0-a7ca-a8d8fe46fc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:54.871106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4dad3-49f8-4dc0-a7ca-a8d8fe46fc38 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:54.871389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4dad3-49f8-4dc0-a7ca-a8d8fe46fc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:54.871666 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4dad3-49f8-4dc0-a7ca-a8d8fe46fc38 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:54.872344 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4dad3-49f8-4dc0-a7ca-a8d8fe46fc38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:54.876615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765efc9-eb9b-443d-aa4b-fb5d9ad6bf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:54.876866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4dad3-49f8-4dc0-a7ca-a8d8fe46fc38 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:54.879659 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765efc9-eb9b-443d-aa4b-fb5d9ad6bf29 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:54.880110 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765efc9-eb9b-443d-aa4b-fb5d9ad6bf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:54.880518 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765efc9-eb9b-443d-aa4b-fb5d9ad6bf29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:54.881865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765efc9-eb9b-443d-aa4b-fb5d9ad6bf29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:54.885064 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765efc9-eb9b-443d-aa4b-fb5d9ad6bf29 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:54.909540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 800/1590] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:25:54 2022] POST /identity/v3/users => generated 297 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:54.909540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4dad3-49f8-4dc0-a7ca-a8d8fe46fc38 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:54.920894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4dad3-49f8-4dc0-a7ca-a8d8fe46fc38 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:54.926785 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39925ac1-2a74-49d2-9006-797499d6b43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:54.927455 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39925ac1-2a74-49d2-9006-797499d6b43e admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:54.927766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39925ac1-2a74-49d2-9006-797499d6b43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:54.928146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39925ac1-2a74-49d2-9006-797499d6b43e admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:54.928896 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39925ac1-2a74-49d2-9006-797499d6b43e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:54.931100 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39925ac1-2a74-49d2-9006-797499d6b43e admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:54.949498 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 791/1591] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:25:54 2022] POST /identity/v3/projects => generated 367 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:54.955601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765efc9-eb9b-443d-aa4b-fb5d9ad6bf29 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:54.967566 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 801/1592] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:25:54 2022] POST /identity/v3/users => generated 292 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:54.968167 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39925ac1-2a74-49d2-9006-797499d6b43e admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:54.979419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39925ac1-2a74-49d2-9006-797499d6b43e admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:55.052108 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765efc9-eb9b-443d-aa4b-fb5d9ad6bf29 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:55.058081 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446eade5-483f-462f-9410-5ddad81a5bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:55.059247 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446eade5-483f-462f-9410-5ddad81a5bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:55.059749 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446eade5-483f-462f-9410-5ddad81a5bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:55.060264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446eade5-483f-462f-9410-5ddad81a5bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:55.061158 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446eade5-483f-462f-9410-5ddad81a5bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:55.063428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446eade5-483f-462f-9410-5ddad81a5bb8 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:55.065964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b091e20-4b86-4376-acad-3bf8ffd499fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:55.066616 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b091e20-4b86-4376-acad-3bf8ffd499fb admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:55.066923 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b091e20-4b86-4376-acad-3bf8ffd499fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:55.067703 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b091e20-4b86-4376-acad-3bf8ffd499fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:55.069252 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b091e20-4b86-4376-acad-3bf8ffd499fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:55.071017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b091e20-4b86-4376-acad-3bf8ffd499fb admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:55.082278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 792/1593] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:55.084280 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2fecf-4441-4668-9265-e2cc807277c9 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:55.084907 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2fecf-4441-4668-9265-e2cc807277c9 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:55.085395 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 802/1594] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:55.085673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2fecf-4441-4668-9265-e2cc807277c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:55.086357 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b091e20-4b86-4376-acad-3bf8ffd499fb admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:55.098481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b091e20-4b86-4376-acad-3bf8ffd499fb admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:55.109593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d9d7a-fe86-41c2-a02c-2c379bb394d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:55.110636 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d9d7a-fe86-41c2-a02c-2c379bb394d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:55.111429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d9d7a-fe86-41c2-a02c-2c379bb394d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:55.111983 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-89e2fecf-4441-4668-9265-e2cc807277c9 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:55.112604 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d9d7a-fe86-41c2-a02c-2c379bb394d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:55.113286 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-89e2fecf-4441-4668-9265-e2cc807277c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:55.113958 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d9d7a-fe86-41c2-a02c-2c379bb394d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:55.116185 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d9d7a-fe86-41c2-a02c-2c379bb394d8 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:55.123848 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 793/1595] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:55.125254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2fecf-4441-4668-9265-e2cc807277c9 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:55.139570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 803/1596] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:55.141044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d9d7a-fe86-41c2-a02c-2c379bb394d8 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:55.159857 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d9d7a-fe86-41c2-a02c-2c379bb394d8 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:55.163670 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2fecf-4441-4668-9265-e2cc807277c9 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:55.167571 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da591556-b896-4e0e-9145-eae5c04b5b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:55.168587 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da591556-b896-4e0e-9145-eae5c04b5b7f admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:55.169072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da591556-b896-4e0e-9145-eae5c04b5b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:55.169521 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da591556-b896-4e0e-9145-eae5c04b5b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:55.170366 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da591556-b896-4e0e-9145-eae5c04b5b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:55.175348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930557f0-c519-463c-85c3-be4bda10ce44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:55.175905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da591556-b896-4e0e-9145-eae5c04b5b7f admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:55.176714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930557f0-c519-463c-85c3-be4bda10ce44 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:55.177194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930557f0-c519-463c-85c3-be4bda10ce44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:55.178183 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930557f0-c519-463c-85c3-be4bda10ce44 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:55.179562 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930557f0-c519-463c-85c3-be4bda10ce44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:55.184195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930557f0-c519-463c-85c3-be4bda10ce44 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:55.194610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 804/1597] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:55.196097 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da591556-b896-4e0e-9145-eae5c04b5b7f admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:55.211033 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da591556-b896-4e0e-9145-eae5c04b5b7f admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:55.219740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955a5f4-b9d5-4fa9-8954-8612f85ba74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:55.220455 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955a5f4-b9d5-4fa9-8954-8612f85ba74e admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:55.221107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955a5f4-b9d5-4fa9-8954-8612f85ba74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:55.221361 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955a5f4-b9d5-4fa9-8954-8612f85ba74e admin admin] PATH_INFO: `/v3/projects/a0d4f26ba74c426abe9161846d0c4e0c/users/e797e79cb6e5428098b85fc328df043a/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:55.237861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 794/1598] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:25:55 2022] POST /identity/v3/users => generated 284 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:55.237861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930557f0-c519-463c-85c3-be4bda10ce44 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:55.242124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955a5f4-b9d5-4fa9-8954-8612f85ba74e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d4f26ba74c426abe9161846d0c4e0c, user_id=e797e79cb6e5428098b85fc328df043a, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:55.246032 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955a5f4-b9d5-4fa9-8954-8612f85ba74e admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:55.249633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930557f0-c519-463c-85c3-be4bda10ce44 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:55.257136 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a095654-4178-48ba-b497-b3025e0105c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:55.259156 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095654-4178-48ba-b497-b3025e0105c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:55.259440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095654-4178-48ba-b497-b3025e0105c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:55.259717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095654-4178-48ba-b497-b3025e0105c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:55.261866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a095654-4178-48ba-b497-b3025e0105c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:55.265160 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a095654-4178-48ba-b497-b3025e0105c6 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:55.271283 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 805/1599] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:25:55 2022] PUT /identity/v3/projects/a0d4f26ba74c426abe9161846d0c4e0c/users/e797e79cb6e5428098b85fc328df043a/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:25:55.272179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955a5f4-b9d5-4fa9-8954-8612f85ba74e admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:55.286785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 795/1600] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:55.291903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a095654-4178-48ba-b497-b3025e0105c6 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:55.301967 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:55.306997 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a095654-4178-48ba-b497-b3025e0105c6 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:55.390803 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:55.411322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef50faa-aabc-4235-b729-9d87add003b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:55.412125 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef50faa-aabc-4235-b729-9d87add003b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:55.412649 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef50faa-aabc-4235-b729-9d87add003b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:55.413170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef50faa-aabc-4235-b729-9d87add003b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:55.414156 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef50faa-aabc-4235-b729-9d87add003b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:55.416988 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef50faa-aabc-4235-b729-9d87add003b3 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:55.430343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955a5f4-b9d5-4fa9-8954-8612f85ba74e admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:55.436965 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590de75-0715-47c3-bd7d-f07417c23339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:55.438033 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590de75-0715-47c3-bd7d-f07417c23339 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:55.438492 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590de75-0715-47c3-bd7d-f07417c23339 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:55.438974 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 796/1601] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:55.439138 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590de75-0715-47c3-bd7d-f07417c23339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:55.441124 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef50faa-aabc-4235-b729-9d87add003b3 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:55.455148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef50faa-aabc-4235-b729-9d87add003b3 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:55.461208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590de75-0715-47c3-bd7d-f07417c23339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:55.464542 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590de75-0715-47c3-bd7d-f07417c23339 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:55.465890 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cfda6-c635-4893-ac63-a922963981e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:55.468439 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cfda6-c635-4893-ac63-a922963981e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:55.468950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cfda6-c635-4893-ac63-a922963981e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:55.469495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cfda6-c635-4893-ac63-a922963981e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:55.471766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161cfda6-c635-4893-ac63-a922963981e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:55.477080 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161cfda6-c635-4893-ac63-a922963981e7 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:55.482790 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 806/1602] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:25:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:25:55.484028 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590de75-0715-47c3-bd7d-f07417c23339 service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:55.497396 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590de75-0715-47c3-bd7d-f07417c23339 service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:55.502731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334d94b-88e9-452c-9f0b-081429aa98e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:55.503364 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334d94b-88e9-452c-9f0b-081429aa98e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:55.503640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334d94b-88e9-452c-9f0b-081429aa98e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:55.504137 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334d94b-88e9-452c-9f0b-081429aa98e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:55.504854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334d94b-88e9-452c-9f0b-081429aa98e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:55.506561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7334d94b-88e9-452c-9f0b-081429aa98e6 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:55.525489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 797/1603] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:55.527671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45569e-b901-487a-890a-8dfcc0434f2a None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:55.527975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45569e-b901-487a-890a-8dfcc0434f2a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:55.528317 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45569e-b901-487a-890a-8dfcc0434f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:55.537365 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 807/1604] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:55.537365 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334d94b-88e9-452c-9f0b-081429aa98e6 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:55.568302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334d94b-88e9-452c-9f0b-081429aa98e6 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:55.583546 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578ef5c-f099-45b3-87be-064b8b827412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:55.584388 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578ef5c-f099-45b3-87be-064b8b827412 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:55.584669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578ef5c-f099-45b3-87be-064b8b827412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:55.586301 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578ef5c-f099-45b3-87be-064b8b827412 admin admin] PATH_INFO: `/v3/projects/493acaa0d6944f94a0dbae8b55b33b88/users/a51a34944fef4fb284d4d835ec03fb61/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:55.627450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-1a45569e-b901-487a-890a-8dfcc0434f2a None None] MFA Rules not processed for user `e797e79cb6e5428098b85fc328df043a`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:55.630162 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1a45569e-b901-487a-890a-8dfcc0434f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:55.643759 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578ef5c-f099-45b3-87be-064b8b827412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=493acaa0d6944f94a0dbae8b55b33b88, user_id=a51a34944fef4fb284d4d835ec03fb61, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:55.650638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578ef5c-f099-45b3-87be-064b8b827412 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:55.739169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 808/1605] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:25:55 2022] PUT /identity/v3/projects/493acaa0d6944f94a0dbae8b55b33b88/users/a51a34944fef4fb284d4d835ec03fb61/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:25:55.741429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578ef5c-f099-45b3-87be-064b8b827412 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:55.850432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 798/1606] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:55.851545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45569e-b901-487a-890a-8dfcc0434f2a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:55.881224 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45569e-b901-487a-890a-8dfcc0434f2a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:55.932068 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578ef5c-f099-45b3-87be-064b8b827412 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:55.942244 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6cc07-c885-41de-8811-615d6a49b6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:55.943195 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6cc07-c885-41de-8811-615d6a49b6a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:55.943689 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6cc07-c885-41de-8811-615d6a49b6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:55.944773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6cc07-c885-41de-8811-615d6a49b6a3 admin admin] PATH_INFO: `/v3/projects/0fe60f1637744c3f8a25e83f48bfdf71/users/f643fd4ee6bc4ccfae45f3b8523ae46a/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:56.044659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6cc07-c885-41de-8811-615d6a49b6a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe60f1637744c3f8a25e83f48bfdf71, user_id=f643fd4ee6bc4ccfae45f3b8523ae46a, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:56.044659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6cc07-c885-41de-8811-615d6a49b6a3 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:56.044659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 809/1607] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:25:55 2022] PUT /identity/v3/projects/0fe60f1637744c3f8a25e83f48bfdf71/users/f643fd4ee6bc4ccfae45f3b8523ae46a/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:25:56.044659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6cc07-c885-41de-8811-615d6a49b6a3 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:56.094935 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bd2b5-6e70-4af4-a6db-90dfee14c737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:56.096290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bd2b5-6e70-4af4-a6db-90dfee14c737 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:56.096935 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bd2b5-6e70-4af4-a6db-90dfee14c737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:56.097909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bd2b5-6e70-4af4-a6db-90dfee14c737 admin admin] PATH_INFO: `/v3/projects/24dc0fdb55324beca6e1af076c0fcd20/users/ddb92150d9be4326b613600de6aeb66b/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:56.122735 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bd2b5-6e70-4af4-a6db-90dfee14c737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24dc0fdb55324beca6e1af076c0fcd20, user_id=ddb92150d9be4326b613600de6aeb66b, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:56.128095 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bd2b5-6e70-4af4-a6db-90dfee14c737 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:56.166061 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 799/1608] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:25:55 2022] PUT /identity/v3/projects/24dc0fdb55324beca6e1af076c0fcd20/users/ddb92150d9be4326b613600de6aeb66b/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:25:56.183637 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1f9d3-31a3-47bf-bed0-3913f5761b8a None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:56.183877 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1f9d3-31a3-47bf-bed0-3913f5761b8a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:56.190411 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1f9d3-31a3-47bf-bed0-3913f5761b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:56.205117 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:56.299872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6cc07-c885-41de-8811-615d6a49b6a3 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:56.303052 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-a2e1f9d3-31a3-47bf-bed0-3913f5761b8a None None] MFA Rules not processed for user `a51a34944fef4fb284d4d835ec03fb61`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:56.305547 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-a2e1f9d3-31a3-47bf-bed0-3913f5761b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:56.481825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 800/1609] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:56.487105 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e1f9d3-31a3-47bf-bed0-3913f5761b8a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:56.488916 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:56.556232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d08e35-3660-4e4d-950a-e4d9c3612230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:56.557638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d08e35-3660-4e4d-950a-e4d9c3612230 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:56.558181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d08e35-3660-4e4d-950a-e4d9c3612230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:56.558491 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d08e35-3660-4e4d-950a-e4d9c3612230 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:56.561342 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d08e35-3660-4e4d-950a-e4d9c3612230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:56.562591 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d08e35-3660-4e4d-950a-e4d9c3612230 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:56.580643 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 810/1610] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 538 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:56.582361 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5e53e-0b06-4788-bacc-9d569318f629 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:56.582661 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5e53e-0b06-4788-bacc-9d569318f629 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:56.582998 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5e53e-0b06-4788-bacc-9d569318f629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:56.612283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e1f9d3-31a3-47bf-bed0-3913f5761b8a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:56.620839 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44107ba3-4a2f-4969-86da-70812d65630f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:56.621949 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44107ba3-4a2f-4969-86da-70812d65630f admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:56.622328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44107ba3-4a2f-4969-86da-70812d65630f admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:56.622710 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44107ba3-4a2f-4969-86da-70812d65630f admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:56.623497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44107ba3-4a2f-4969-86da-70812d65630f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:56.625787 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44107ba3-4a2f-4969-86da-70812d65630f admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:56.637271 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-04c5e53e-0b06-4788-bacc-9d569318f629 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:56.639463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-04c5e53e-0b06-4788-bacc-9d569318f629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:56.661657 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 811/1611] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:56.663276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988a093-ba29-4ec7-a846-0a790aeede7a None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:56.663634 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988a093-ba29-4ec7-a846-0a790aeede7a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:56.663928 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988a093-ba29-4ec7-a846-0a790aeede7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:56.702301 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 801/1612] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:25:56 2022] POST /identity/v3/projects => generated 365 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:56.702301 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44107ba3-4a2f-4969-86da-70812d65630f admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:56.710547 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-a988a093-ba29-4ec7-a846-0a790aeede7a None None] MFA Rules not processed for user `ddb92150d9be4326b613600de6aeb66b`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:56.711917 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a988a093-ba29-4ec7-a846-0a790aeede7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:56.857278 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 812/1613] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:56.859226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988a093-ba29-4ec7-a846-0a790aeede7a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:57.009630 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988a093-ba29-4ec7-a846-0a790aeede7a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:57.011237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44107ba3-4a2f-4969-86da-70812d65630f admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:57.023618 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33017884-6e5d-49ea-b3cc-6645ed9961e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:57.029658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c990c-de49-4f93-b3f1-dbe84d23a6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:57.031389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c990c-de49-4f93-b3f1-dbe84d23a6ca admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:57.031389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c990c-de49-4f93-b3f1-dbe84d23a6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:57.031742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c990c-de49-4f93-b3f1-dbe84d23a6ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:57.033143 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c990c-de49-4f93-b3f1-dbe84d23a6ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:57.036019 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33017884-6e5d-49ea-b3cc-6645ed9961e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:57.038316 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33017884-6e5d-49ea-b3cc-6645ed9961e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:57.038647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c990c-de49-4f93-b3f1-dbe84d23a6ca admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:57.039179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33017884-6e5d-49ea-b3cc-6645ed9961e1 admin admin] PATH_INFO: `/v3/projects/7adc672f1cfd4f31a962a47097b7d073/users/d536120c684e405fb96766b87764ec9c/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:57.082170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33017884-6e5d-49ea-b3cc-6645ed9961e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7adc672f1cfd4f31a962a47097b7d073, user_id=d536120c684e405fb96766b87764ec9c, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:57.092876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33017884-6e5d-49ea-b3cc-6645ed9961e1 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:57.120704 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 802/1614] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:25:56 2022] POST /identity/v3/projects => generated 381 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:57.121718 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c990c-de49-4f93-b3f1-dbe84d23a6ca admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:57.132425 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:57.146271 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 813/1615] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:25:56 2022] PUT /identity/v3/projects/7adc672f1cfd4f31a962a47097b7d073/users/d536120c684e405fb96766b87764ec9c/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:25:57.149714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33017884-6e5d-49ea-b3cc-6645ed9961e1 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:57.268058 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c990c-de49-4f93-b3f1-dbe84d23a6ca admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:57.273821 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3d8a0-0b52-486a-bc00-88658e69e912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:57.274871 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3d8a0-0b52-486a-bc00-88658e69e912 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:57.275352 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3d8a0-0b52-486a-bc00-88658e69e912 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:57.275814 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3d8a0-0b52-486a-bc00-88658e69e912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:57.283741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:57.324987 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33017884-6e5d-49ea-b3cc-6645ed9961e1 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:57.416182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3d8a0-0b52-486a-bc00-88658e69e912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:57.418464 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3d8a0-0b52-486a-bc00-88658e69e912 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:57.432328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 803/1616] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:25:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:25:57.434069 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7df2f-e26b-4d6e-9557-cd7e21e7f335 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:57.434755 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7df2f-e26b-4d6e-9557-cd7e21e7f335 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:57.435266 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7df2f-e26b-4d6e-9557-cd7e21e7f335 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:57.436221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d15097-c9c1-4709-954a-038ca5bcd456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:57.436931 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d15097-c9c1-4709-954a-038ca5bcd456 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:57.437268 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d15097-c9c1-4709-954a-038ca5bcd456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:57.437690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d15097-c9c1-4709-954a-038ca5bcd456 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:57.438316 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d15097-c9c1-4709-954a-038ca5bcd456 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:57.442215 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d15097-c9c1-4709-954a-038ca5bcd456 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:57.443111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 804/1617] 10.222.0.17 () {54 vars in 962 bytes} [Sun Aug 7 19:25:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:25:57.444282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7df2f-e26b-4d6e-9557-cd7e21e7f335 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:57.458691 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7df2f-e26b-4d6e-9557-cd7e21e7f335 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:57.464774 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db190573-2418-4b14-a599-79e8e8ddd290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:57.466193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db190573-2418-4b14-a599-79e8e8ddd290 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:57.466741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db190573-2418-4b14-a599-79e8e8ddd290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:57.467214 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db190573-2418-4b14-a599-79e8e8ddd290 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:57.468598 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db190573-2418-4b14-a599-79e8e8ddd290 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:57.471034 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db190573-2418-4b14-a599-79e8e8ddd290 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:57.499955 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 814/1618] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:25:57 2022] POST /identity/v3/users => generated 289 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:57.501261 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5b1e1-d82c-41be-8061-7843ba1e7812 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:57.501550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5b1e1-d82c-41be-8061-7843ba1e7812 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:57.501874 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5b1e1-d82c-41be-8061-7843ba1e7812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:57.516614 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 805/1619] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:25:57 2022] POST /identity/v3/users => generated 297 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:25:57.521138 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eac6e-e845-4445-9af9-8c016c160027 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:57.523072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eac6e-e845-4445-9af9-8c016c160027 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:57.523542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eac6e-e845-4445-9af9-8c016c160027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:57.558265 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-09e5b1e1-d82c-41be-8061-7843ba1e7812 None None] MFA Rules not processed for user `d536120c684e405fb96766b87764ec9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:57.559935 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-09e5b1e1-d82c-41be-8061-7843ba1e7812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:57.595278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-6b3eac6e-e845-4445-9af9-8c016c160027 None None] MFA Rules not processed for user `a4946aec299f45bd8402cff3159d04d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:57.598030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6b3eac6e-e845-4445-9af9-8c016c160027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:57.677361 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 815/1620] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:57.682165 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5b1e1-d82c-41be-8061-7843ba1e7812 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:57.694456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5b1e1-d82c-41be-8061-7843ba1e7812 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:57.700173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee132ee4-5489-4bf6-ac59-3dcae086f865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:57.700836 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee132ee4-5489-4bf6-ac59-3dcae086f865 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:57.701103 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee132ee4-5489-4bf6-ac59-3dcae086f865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:57.701380 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee132ee4-5489-4bf6-ac59-3dcae086f865 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:57.702049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee132ee4-5489-4bf6-ac59-3dcae086f865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:57.703738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee132ee4-5489-4bf6-ac59-3dcae086f865 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:57.716502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 816/1621] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:57.717641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee132ee4-5489-4bf6-ac59-3dcae086f865 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:57.729033 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee132ee4-5489-4bf6-ac59-3dcae086f865 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:57.735473 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142753a-05e0-45e6-8b97-f9de70e26a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:57.736442 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142753a-05e0-45e6-8b97-f9de70e26a60 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:57.736669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142753a-05e0-45e6-8b97-f9de70e26a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:57.737065 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142753a-05e0-45e6-8b97-f9de70e26a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:57.737766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142753a-05e0-45e6-8b97-f9de70e26a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:57.740746 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142753a-05e0-45e6-8b97-f9de70e26a60 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:57.754124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 817/1622] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:57.755081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142753a-05e0-45e6-8b97-f9de70e26a60 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:57.757757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 806/1623] 10.222.0.17 () {58 vars in 1059 bytes} [Sun Aug 7 19:25:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:57.766991 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3eac6e-e845-4445-9af9-8c016c160027 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:57.794315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3eac6e-e845-4445-9af9-8c016c160027 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:57.809698 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1a980-a4a1-4806-a42b-7d5d9ab1e373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:57.810411 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1a980-a4a1-4806-a42b-7d5d9ab1e373 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:57.810695 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1a980-a4a1-4806-a42b-7d5d9ab1e373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:57.810992 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1a980-a4a1-4806-a42b-7d5d9ab1e373 admin admin] PATH_INFO: `/v3/projects/63b6dbd6c3aa4c2f927c20c81d26de0c/users/422e6e0a83474cb49d2770c73ed59e46/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:57.817473 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:57.831680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142753a-05e0-45e6-8b97-f9de70e26a60 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:57.835349 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:57.837484 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d428d-02cd-4898-abfa-61eb7976ec41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:57.838300 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d428d-02cd-4898-abfa-61eb7976ec41 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:57.838515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d428d-02cd-4898-abfa-61eb7976ec41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:57.839142 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d428d-02cd-4898-abfa-61eb7976ec41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:57.839737 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c1a980-a4a1-4806-a42b-7d5d9ab1e373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63b6dbd6c3aa4c2f927c20c81d26de0c, user_id=422e6e0a83474cb49d2770c73ed59e46, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:57.842566 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c1a980-a4a1-4806-a42b-7d5d9ab1e373 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:57.854789 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d428d-02cd-4898-abfa-61eb7976ec41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:57.856492 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d428d-02cd-4898-abfa-61eb7976ec41 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:57.882208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 807/1624] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:25:57 2022] PUT /identity/v3/projects/63b6dbd6c3aa4c2f927c20c81d26de0c/users/422e6e0a83474cb49d2770c73ed59e46/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:25:57.883176 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1a980-a4a1-4806-a42b-7d5d9ab1e373 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:57.886254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 818/1625] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:25:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:25:57.888060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d428d-02cd-4898-abfa-61eb7976ec41 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:57.921309 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d428d-02cd-4898-abfa-61eb7976ec41 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:58.093377 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1a980-a4a1-4806-a42b-7d5d9ab1e373 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:58.106680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294c539-0287-4ed4-be0e-664968018861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:58.107386 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294c539-0287-4ed4-be0e-664968018861 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:58.111349 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294c539-0287-4ed4-be0e-664968018861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:58.111736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294c539-0287-4ed4-be0e-664968018861 admin admin] PATH_INFO: `/v3/projects/babe6e4cd9a94181b5b42f4927aa98a8/users/50fb339b2fa74f9192730d34df80a1a1/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:58.149615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7ddf5-83d8-434e-a045-0377bd33aeeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:58.151476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7ddf5-83d8-434e-a045-0377bd33aeeb service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:58.151755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7ddf5-83d8-434e-a045-0377bd33aeeb service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:58.152054 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7ddf5-83d8-434e-a045-0377bd33aeeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:58.166977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294c539-0287-4ed4-be0e-664968018861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=babe6e4cd9a94181b5b42f4927aa98a8, user_id=50fb339b2fa74f9192730d34df80a1a1, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:58.172814 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294c539-0287-4ed4-be0e-664968018861 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:58.204711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 808/1626] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:25:57 2022] PUT /identity/v3/projects/babe6e4cd9a94181b5b42f4927aa98a8/users/50fb339b2fa74f9192730d34df80a1a1/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:25:58.205965 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294c539-0287-4ed4-be0e-664968018861 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:58.241276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7ddf5-83d8-434e-a045-0377bd33aeeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:58.244488 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7ddf5-83d8-434e-a045-0377bd33aeeb service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:58.273662 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 819/1627] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:25:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:25:58.296114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7ddf5-83d8-434e-a045-0377bd33aeeb service nova] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:58.368887 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7ddf5-83d8-434e-a045-0377bd33aeeb service nova] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:58.497762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294c539-0287-4ed4-be0e-664968018861 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:58.512935 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bfcf5-369a-4cd0-b39b-06aaa9025596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:58.513589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bfcf5-369a-4cd0-b39b-06aaa9025596 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:58.513870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bfcf5-369a-4cd0-b39b-06aaa9025596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:58.514150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21bfcf5-369a-4cd0-b39b-06aaa9025596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:58.514817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bfcf5-369a-4cd0-b39b-06aaa9025596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:58.519441 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21bfcf5-369a-4cd0-b39b-06aaa9025596 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:58.553360 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 809/1628] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:58.559003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bfcf5-369a-4cd0-b39b-06aaa9025596 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:58.591602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21bfcf5-369a-4cd0-b39b-06aaa9025596 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:58.617517 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f772fc8-312c-430e-8354-0dbe532d3fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:58.625577 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f772fc8-312c-430e-8354-0dbe532d3fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:58.626159 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f772fc8-312c-430e-8354-0dbe532d3fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:58.626644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f772fc8-312c-430e-8354-0dbe532d3fbb admin admin] PATH_INFO: `/v3/projects/63b6dbd6c3aa4c2f927c20c81d26de0c/users/422e6e0a83474cb49d2770c73ed59e46/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:58.639297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f772fc8-312c-430e-8354-0dbe532d3fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=63b6dbd6c3aa4c2f927c20c81d26de0c, user_id=422e6e0a83474cb49d2770c73ed59e46, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:58.653569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f772fc8-312c-430e-8354-0dbe532d3fbb admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:58.660429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7c3b7-cb68-478b-95a1-2d277e0b9ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:58.668163 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7c3b7-cb68-478b-95a1-2d277e0b9ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:58.670877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7c3b7-cb68-478b-95a1-2d277e0b9ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:58.674221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7c3b7-cb68-478b-95a1-2d277e0b9ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:58.679796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7c3b7-cb68-478b-95a1-2d277e0b9ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:58.692260 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7c3b7-cb68-478b-95a1-2d277e0b9ce8 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:58.711556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 810/1629] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:25:58 2022] PUT /identity/v3/projects/63b6dbd6c3aa4c2f927c20c81d26de0c/users/422e6e0a83474cb49d2770c73ed59e46/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:25:58.722893 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5b4ac-ca6f-4931-9e1e-40df52b59bf9 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:58.723204 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5b4ac-ca6f-4931-9e1e-40df52b59bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:58.723664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5b4ac-ca6f-4931-9e1e-40df52b59bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:58.744828 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 820/1630] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:25:58.752536 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7c3b7-cb68-478b-95a1-2d277e0b9ce8 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:58.778087 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7c3b7-cb68-478b-95a1-2d277e0b9ce8 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:58.813121 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:58.842803 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-16a5b4ac-ca6f-4931-9e1e-40df52b59bf9 None None] MFA Rules not processed for user `422e6e0a83474cb49d2770c73ed59e46`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:58.844132 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-16a5b4ac-ca6f-4931-9e1e-40df52b59bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:58.989167 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e8b6a5-c4b5-4ff6-a885-2805740dd43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:58.990400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8b6a5-c4b5-4ff6-a885-2805740dd43a admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:58.991160 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8b6a5-c4b5-4ff6-a885-2805740dd43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:58.992258 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e8b6a5-c4b5-4ff6-a885-2805740dd43a admin admin] PATH_INFO: `/v3/projects/babe6e4cd9a94181b5b42f4927aa98a8/users/50fb339b2fa74f9192730d34df80a1a1/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:58.996473 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8b6a5-c4b5-4ff6-a885-2805740dd43a admin admin] RBAC: Authorizing `identity:create_grant(project_id=babe6e4cd9a94181b5b42f4927aa98a8, user_id=50fb339b2fa74f9192730d34df80a1a1, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:59.004356 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e8b6a5-c4b5-4ff6-a885-2805740dd43a admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:59.019928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:25:59.079120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 821/1631] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:25:58 2022] PUT /identity/v3/projects/babe6e4cd9a94181b5b42f4927aa98a8/users/50fb339b2fa74f9192730d34df80a1a1/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:25:59.086797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb5044-0d20-404c-a66d-c6cc8a55807f None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:59.087587 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb5044-0d20-404c-a66d-c6cc8a55807f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:59.088646 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb5044-0d20-404c-a66d-c6cc8a55807f None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:59.097469 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 811/1632] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:58 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:25:59.099119 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66783a-4c1f-46ce-b2e1-16d7844117ee None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:59.099577 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66783a-4c1f-46ce-b2e1-16d7844117ee None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:59.099902 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66783a-4c1f-46ce-b2e1-16d7844117ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:59.103693 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 822/1633] 10.222.0.17 () {54 vars in 912 bytes} [Sun Aug 7 19:25:59 2022] GET /identity => generated 270 bytes in 24 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:25:59.110289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb5044-0d20-404c-a66d-c6cc8a55807f None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:59.187119 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-ee66783a-4c1f-46ce-b2e1-16d7844117ee None None] MFA Rules not processed for user `50fb339b2fa74f9192730d34df80a1a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:59.188422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ee66783a-4c1f-46ce-b2e1-16d7844117ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:59.308898 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb5044-0d20-404c-a66d-c6cc8a55807f None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:59.317150 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491e06d-31be-4ce8-93e1-df8f7dd990b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:59.318106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491e06d-31be-4ce8-93e1-df8f7dd990b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:59.318535 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491e06d-31be-4ce8-93e1-df8f7dd990b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:59.318975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491e06d-31be-4ce8-93e1-df8f7dd990b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:59.496407 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 812/1634] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:25:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:59.496407 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af502e7a-cbab-433d-9169-e7832456561f None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:59.496407 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af502e7a-cbab-433d-9169-e7832456561f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:59.496407 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af502e7a-cbab-433d-9169-e7832456561f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:59.552278 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491e06d-31be-4ce8-93e1-df8f7dd990b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:59.553498 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491e06d-31be-4ce8-93e1-df8f7dd990b9 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:59.558724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-af502e7a-cbab-433d-9169-e7832456561f None None] MFA Rules not processed for user `a4946aec299f45bd8402cff3159d04d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:25:59.560304 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-af502e7a-cbab-433d-9169-e7832456561f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:25:59.580641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 823/1635] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:25:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:25:59.581300 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491e06d-31be-4ce8-93e1-df8f7dd990b9 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:59.618789 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491e06d-31be-4ce8-93e1-df8f7dd990b9 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:59.639243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44d333-989b-493a-9f95-bb7b8ce6df72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:59.640030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44d333-989b-493a-9f95-bb7b8ce6df72 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:59.640258 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44d333-989b-493a-9f95-bb7b8ce6df72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:59.640465 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44d333-989b-493a-9f95-bb7b8ce6df72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:59.674857 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c44d333-989b-493a-9f95-bb7b8ce6df72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:59.676411 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c44d333-989b-493a-9f95-bb7b8ce6df72 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:59.707602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 824/1636] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:25:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:25:59.772757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 813/1637] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Aug 7 19:25:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:25:59.867672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44d333-989b-493a-9f95-bb7b8ce6df72 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:25:59.878523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44d333-989b-493a-9f95-bb7b8ce6df72 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:25:59.886580 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb234c-8382-4730-a816-f3535814f6c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:25:59.889227 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb234c-8382-4730-a816-f3535814f6c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:25:59.890123 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb234c-8382-4730-a816-f3535814f6c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:25:59.892270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb234c-8382-4730-a816-f3535814f6c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:25:59.920732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb234c-8382-4730-a816-f3535814f6c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:25:59.922564 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb234c-8382-4730-a816-f3535814f6c7 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:25:59.952957 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 825/1638] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:25:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:25:59.993396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af502e7a-cbab-433d-9169-e7832456561f None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:00.007800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af502e7a-cbab-433d-9169-e7832456561f None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:00.017137 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb16f68-e79b-4734-b5fc-1d40bd3ddc02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:00.018079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb16f68-e79b-4734-b5fc-1d40bd3ddc02 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:00.018366 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb16f68-e79b-4734-b5fc-1d40bd3ddc02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:00.018810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb16f68-e79b-4734-b5fc-1d40bd3ddc02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:00.157357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:00.164203 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb16f68-e79b-4734-b5fc-1d40bd3ddc02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:00.165757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb16f68-e79b-4734-b5fc-1d40bd3ddc02 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:00.194153 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 814/1639] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:25:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:01.521804 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fb400-7d3e-472f-af35-01f82533f6d3 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:01.522732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fb400-7d3e-472f-af35-01f82533f6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:01.523165 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64fb400-7d3e-472f-af35-01f82533f6d3 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:01.525525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 826/1640] 10.222.0.17 () {54 vars in 962 bytes} [Sun Aug 7 19:26:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:26:01.544135 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a181d6-3e80-4cbb-ac25-aebea6961e25 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:01.545015 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a181d6-3e80-4cbb-ac25-aebea6961e25 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:01.545453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a181d6-3e80-4cbb-ac25-aebea6961e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:01.601054 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-95a181d6-3e80-4cbb-ac25-aebea6961e25 None None] MFA Rules not processed for user `a4946aec299f45bd8402cff3159d04d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:01.604338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-95a181d6-3e80-4cbb-ac25-aebea6961e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:01.633760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 815/1641] 10.222.0.17 () {58 vars in 1059 bytes} [Sun Aug 7 19:26:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:01.639292 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0778d-8ca6-4bcb-a413-46aad9acaf35 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:01.639557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0778d-8ca6-4bcb-a413-46aad9acaf35 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:01.639844 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0778d-8ca6-4bcb-a413-46aad9acaf35 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:01.641553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 827/1642] 10.222.0.17 () {54 vars in 912 bytes} [Sun Aug 7 19:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:26:01.647531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d6d19-0353-47cd-8f75-9fb828503dcc None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:01.647822 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d6d19-0353-47cd-8f75-9fb828503dcc None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:01.648141 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d6d19-0353-47cd-8f75-9fb828503dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:01.748904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-ee4d6d19-0353-47cd-8f75-9fb828503dcc None None] MFA Rules not processed for user `a4946aec299f45bd8402cff3159d04d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:01.751355 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ee4d6d19-0353-47cd-8f75-9fb828503dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:01.818772 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 816/1643] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Aug 7 19:26:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:01.912169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0778d-8ca6-4bcb-a413-46aad9acaf35 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:01.961775 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0778d-8ca6-4bcb-a413-46aad9acaf35 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:01.994063 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645876a3-0cb0-4ab1-9c53-ca56ab1a632b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:01.995155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645876a3-0cb0-4ab1-9c53-ca56ab1a632b service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:01.995669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645876a3-0cb0-4ab1-9c53-ca56ab1a632b service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:01.996195 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645876a3-0cb0-4ab1-9c53-ca56ab1a632b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:02.059260 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645876a3-0cb0-4ab1-9c53-ca56ab1a632b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:02.066077 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645876a3-0cb0-4ab1-9c53-ca56ab1a632b service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:02.122083 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 828/1644] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:26:02.931366 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d6d19-0353-47cd-8f75-9fb828503dcc None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:03.154176 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d6d19-0353-47cd-8f75-9fb828503dcc None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:03.183171 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e8686-3021-4c0f-92d8-7ab16a1a444d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:03.187637 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e8686-3021-4c0f-92d8-7ab16a1a444d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:03.188059 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e8686-3021-4c0f-92d8-7ab16a1a444d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:03.188255 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e8686-3021-4c0f-92d8-7ab16a1a444d admin admin] PATH_INFO: `/v3/users/e797e79cb6e5428098b85fc328df043a` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:03.190538 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e8686-3021-4c0f-92d8-7ab16a1a444d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e797e79cb6e5428098b85fc328df043a)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:03.198787 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e8686-3021-4c0f-92d8-7ab16a1a444d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:03.367327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e80e8686-3021-4c0f-92d8-7ab16a1a444d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e797e79cb6e5428098b85fc328df043a'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:03.422493 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e80e8686-3021-4c0f-92d8-7ab16a1a444d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e797e79cb6e5428098b85fc328df043a'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:03.462056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e80e8686-3021-4c0f-92d8-7ab16a1a444d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e797e79cb6e5428098b85fc328df043a'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:03.467474 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e80e8686-3021-4c0f-92d8-7ab16a1a444d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e797e79cb6e5428098b85fc328df043a'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:03.473805 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e80e8686-3021-4c0f-92d8-7ab16a1a444d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e797e79cb6e5428098b85fc328df043a'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:03.547231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 817/1645] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:02 2022] DELETE /identity/v3/users/e797e79cb6e5428098b85fc328df043a => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:03.554492 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da2fe8-d6fa-422d-9947-ce801c2ef270 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:03.555062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da2fe8-d6fa-422d-9947-ce801c2ef270 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:03.555500 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da2fe8-d6fa-422d-9947-ce801c2ef270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:03.658518 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-80da2fe8-d6fa-422d-9947-ce801c2ef270 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:03.663801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-80da2fe8-d6fa-422d-9947-ce801c2ef270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:03.764285 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:03.924630 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 829/1646] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:04.182656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e8686-3021-4c0f-92d8-7ab16a1a444d admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:04.184732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e80e8686-3021-4c0f-92d8-7ab16a1a444d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:04.351363 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e8686-3021-4c0f-92d8-7ab16a1a444d admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:04.366624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294446b-0a60-4f3e-a841-cbaff1ec22f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:04.371171 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294446b-0a60-4f3e-a841-cbaff1ec22f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:04.372367 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294446b-0a60-4f3e-a841-cbaff1ec22f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:04.376748 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294446b-0a60-4f3e-a841-cbaff1ec22f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:04.414678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294446b-0a60-4f3e-a841-cbaff1ec22f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:04.416517 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294446b-0a60-4f3e-a841-cbaff1ec22f5 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:04.453206 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 818/1647] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:06.053135 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da2fe8-d6fa-422d-9947-ce801c2ef270 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:06.055517 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-80da2fe8-d6fa-422d-9947-ce801c2ef270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:06.123319 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da2fe8-d6fa-422d-9947-ce801c2ef270 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:06.146232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ea852d-35f7-467d-9d36-29a403b47e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:06.146910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea852d-35f7-467d-9d36-29a403b47e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:06.147193 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea852d-35f7-467d-9d36-29a403b47e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:06.147470 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ea852d-35f7-467d-9d36-29a403b47e41 admin admin] PATH_INFO: `/v3/projects/a0d4f26ba74c426abe9161846d0c4e0c` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:06.148881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea852d-35f7-467d-9d36-29a403b47e41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0d4f26ba74c426abe9161846d0c4e0c)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:06.150874 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ea852d-35f7-467d-9d36-29a403b47e41 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:06.360866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-c7ea852d-35f7-467d-9d36-29a403b47e41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0d4f26ba74c426abe9161846d0c4e0c', 'request_id': 'req-c7ea852d-35f7-467d-9d36-29a403b47e41'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:06.379721 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-c7ea852d-35f7-467d-9d36-29a403b47e41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0d4f26ba74c426abe9161846d0c4e0c', 'request_id': 'req-c7ea852d-35f7-467d-9d36-29a403b47e41'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:06.424854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-c7ea852d-35f7-467d-9d36-29a403b47e41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0d4f26ba74c426abe9161846d0c4e0c', 'request_id': 'req-c7ea852d-35f7-467d-9d36-29a403b47e41'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:06.470201 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-c7ea852d-35f7-467d-9d36-29a403b47e41 admin admin] The token cache is being invalidate because project a0d4f26ba74c426abe9161846d0c4e0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:06.472239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-c7ea852d-35f7-467d-9d36-29a403b47e41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:06.474325 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-c7ea852d-35f7-467d-9d36-29a403b47e41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:06.482437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 830/1648] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:06 2022] DELETE /identity/v3/projects/a0d4f26ba74c426abe9161846d0c4e0c => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:06.484213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294446b-0a60-4f3e-a841-cbaff1ec22f5 service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:06.494562 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:06.506672 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294446b-0a60-4f3e-a841-cbaff1ec22f5 service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:06.774388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e42dbe-02d0-4172-8f19-e3fcbb4bee16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:06.775602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e42dbe-02d0-4172-8f19-e3fcbb4bee16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:06.776145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e42dbe-02d0-4172-8f19-e3fcbb4bee16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:06.776475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e42dbe-02d0-4172-8f19-e3fcbb4bee16 admin admin] PATH_INFO: `/v3/users/422e6e0a83474cb49d2770c73ed59e46` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:06.779449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e42dbe-02d0-4172-8f19-e3fcbb4bee16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=422e6e0a83474cb49d2770c73ed59e46)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:06.794182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e42dbe-02d0-4172-8f19-e3fcbb4bee16 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:07.063315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-59e42dbe-02d0-4172-8f19-e3fcbb4bee16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '422e6e0a83474cb49d2770c73ed59e46'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:07.102577 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-59e42dbe-02d0-4172-8f19-e3fcbb4bee16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '422e6e0a83474cb49d2770c73ed59e46'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:07.126023 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-59e42dbe-02d0-4172-8f19-e3fcbb4bee16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '422e6e0a83474cb49d2770c73ed59e46'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:07.127697 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-59e42dbe-02d0-4172-8f19-e3fcbb4bee16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '422e6e0a83474cb49d2770c73ed59e46'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:07.128549 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-59e42dbe-02d0-4172-8f19-e3fcbb4bee16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '422e6e0a83474cb49d2770c73ed59e46'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:07.153597 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 819/1649] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:06 2022] DELETE /identity/v3/users/422e6e0a83474cb49d2770c73ed59e46 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:07.609581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb8f2b-380c-435c-8544-69d0beaf3323 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:07.610078 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb8f2b-380c-435c-8544-69d0beaf3323 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:07.610300 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb8f2b-380c-435c-8544-69d0beaf3323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:07.698187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-29fb8f2b-380c-435c-8544-69d0beaf3323 None None] MFA Rules not processed for user `be3e2afc428a45e2b993c39e3d53cd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:07.699756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-29fb8f2b-380c-435c-8544-69d0beaf3323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:07.848456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 831/1650] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:07.855862 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e42dbe-02d0-4172-8f19-e3fcbb4bee16 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:07.857579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-59e42dbe-02d0-4172-8f19-e3fcbb4bee16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:08.096821 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e42dbe-02d0-4172-8f19-e3fcbb4bee16 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:08.106024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb3bf8-c681-487c-888e-738d1448f59e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:08.106959 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb3bf8-c681-487c-888e-738d1448f59e service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:08.107548 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb3bf8-c681-487c-888e-738d1448f59e service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:08.108196 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb3bf8-c681-487c-888e-738d1448f59e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:08.121264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb3bf8-c681-487c-888e-738d1448f59e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:08.123584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb3bf8-c681-487c-888e-738d1448f59e service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:08.151832 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 820/1651] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:09.001327 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb8f2b-380c-435c-8544-69d0beaf3323 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:09.003694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-29fb8f2b-380c-435c-8544-69d0beaf3323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:09.126413 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:09.199235 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb8f2b-380c-435c-8544-69d0beaf3323 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:09.210036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a1ae2-9ade-4c59-b27f-38181bb55526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:09.210727 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a1ae2-9ade-4c59-b27f-38181bb55526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:09.211025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a1ae2-9ade-4c59-b27f-38181bb55526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:09.211276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a1ae2-9ade-4c59-b27f-38181bb55526 admin admin] PATH_INFO: `/v3/projects/63b6dbd6c3aa4c2f927c20c81d26de0c` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:09.212722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a1ae2-9ade-4c59-b27f-38181bb55526 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63b6dbd6c3aa4c2f927c20c81d26de0c)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:09.218507 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a1ae2-9ade-4c59-b27f-38181bb55526 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:09.326977 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-222a1ae2-9ade-4c59-b27f-38181bb55526 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63b6dbd6c3aa4c2f927c20c81d26de0c', 'request_id': 'req-222a1ae2-9ade-4c59-b27f-38181bb55526'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:09.336750 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-222a1ae2-9ade-4c59-b27f-38181bb55526 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63b6dbd6c3aa4c2f927c20c81d26de0c', 'request_id': 'req-222a1ae2-9ade-4c59-b27f-38181bb55526'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:09.345071 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-222a1ae2-9ade-4c59-b27f-38181bb55526 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63b6dbd6c3aa4c2f927c20c81d26de0c', 'request_id': 'req-222a1ae2-9ade-4c59-b27f-38181bb55526'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:09.368609 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-222a1ae2-9ade-4c59-b27f-38181bb55526 admin admin] The token cache is being invalidate because project 63b6dbd6c3aa4c2f927c20c81d26de0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:09.368907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-222a1ae2-9ade-4c59-b27f-38181bb55526 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:09.371913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-222a1ae2-9ade-4c59-b27f-38181bb55526 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:09.380379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 832/1652] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:09 2022] DELETE /identity/v3/projects/63b6dbd6c3aa4c2f927c20c81d26de0c => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:09.389363 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3089504-48bb-4abf-8ab8-384b4267f599 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:09.389923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3089504-48bb-4abf-8ab8-384b4267f599 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:09.390276 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3089504-48bb-4abf-8ab8-384b4267f599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:09.425950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-c3089504-48bb-4abf-8ab8-384b4267f599 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:09.427195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-c3089504-48bb-4abf-8ab8-384b4267f599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:09.432034 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:09.567595 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 821/1653] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:09.576312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4862b-1e8a-4724-a35d-1a9b2dc58b8e None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:09.577249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4862b-1e8a-4724-a35d-1a9b2dc58b8e None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:09.578811 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4862b-1e8a-4724-a35d-1a9b2dc58b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:09.668283 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-25f4862b-1e8a-4724-a35d-1a9b2dc58b8e None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:09.674137 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-25f4862b-1e8a-4724-a35d-1a9b2dc58b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:09.720778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 833/1654] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:09.724996 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3089504-48bb-4abf-8ab8-384b4267f599 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:09.741111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3089504-48bb-4abf-8ab8-384b4267f599 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:09.750896 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34067b8d-50ee-4503-a3a2-f2022aec2f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:09.752293 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34067b8d-50ee-4503-a3a2-f2022aec2f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:09.752859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34067b8d-50ee-4503-a3a2-f2022aec2f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:09.753325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34067b8d-50ee-4503-a3a2-f2022aec2f7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:09.754416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34067b8d-50ee-4503-a3a2-f2022aec2f7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:26:09.755016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34067b8d-50ee-4503-a3a2-f2022aec2f7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:09.757677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34067b8d-50ee-4503-a3a2-f2022aec2f7f admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:09.800510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 822/1655] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:26:09.805129 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4862b-1e8a-4724-a35d-1a9b2dc58b8e None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:09.823737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4862b-1e8a-4724-a35d-1a9b2dc58b8e None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:09.849593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846e8e2-ef6b-460f-8311-2f26738dbe3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:09.851057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846e8e2-ef6b-460f-8311-2f26738dbe3e admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:09.852707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846e8e2-ef6b-460f-8311-2f26738dbe3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:09.855337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846e8e2-ef6b-460f-8311-2f26738dbe3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:09.856303 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846e8e2-ef6b-460f-8311-2f26738dbe3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:09.865068 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1846e8e2-ef6b-460f-8311-2f26738dbe3e admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:09.954606 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 834/1656] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:09 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:26:09.958976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34067b8d-50ee-4503-a3a2-f2022aec2f7f admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:09.972648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34067b8d-50ee-4503-a3a2-f2022aec2f7f admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:10.124187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f4e2d-2b98-47b2-804b-2a51e2419fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:10.124902 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f4e2d-2b98-47b2-804b-2a51e2419fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:10.125186 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f4e2d-2b98-47b2-804b-2a51e2419fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:10.125696 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f4e2d-2b98-47b2-804b-2a51e2419fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:10.126372 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f4e2d-2b98-47b2-804b-2a51e2419fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:10.129082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f4e2d-2b98-47b2-804b-2a51e2419fe2 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:10.193136 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846e8e2-ef6b-460f-8311-2f26738dbe3e admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:10.196856 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 823/1657] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:09 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:10.207539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1846e8e2-ef6b-460f-8311-2f26738dbe3e admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:10.215302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9ffd9-e559-4fce-86ec-17b68007730c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:10.217228 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9ffd9-e559-4fce-86ec-17b68007730c admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:10.217788 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9ffd9-e559-4fce-86ec-17b68007730c admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:10.218241 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9ffd9-e559-4fce-86ec-17b68007730c admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:10.219157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9ffd9-e559-4fce-86ec-17b68007730c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:10.221365 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc9ffd9-e559-4fce-86ec-17b68007730c admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:10.246408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f4e2d-2b98-47b2-804b-2a51e2419fe2 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:10.249168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 835/1658] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:10.261483 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f4e2d-2b98-47b2-804b-2a51e2419fe2 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:10.279613 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1886a-48de-45d5-8270-9f9b6e81674d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:10.280906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1886a-48de-45d5-8270-9f9b6e81674d admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:10.281414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1886a-48de-45d5-8270-9f9b6e81674d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:10.281858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1886a-48de-45d5-8270-9f9b6e81674d admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:10.282679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb1886a-48de-45d5-8270-9f9b6e81674d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:10.287488 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb1886a-48de-45d5-8270-9f9b6e81674d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:10.324320 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9ffd9-e559-4fce-86ec-17b68007730c admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:10.325534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 824/1659] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:10.342028 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9ffd9-e559-4fce-86ec-17b68007730c admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:10.349169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5685c-1d10-46d5-a6bd-856ec45af557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:10.350181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5685c-1d10-46d5-a6bd-856ec45af557 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:10.350556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5685c-1d10-46d5-a6bd-856ec45af557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:10.350938 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5685c-1d10-46d5-a6bd-856ec45af557 admin admin] PATH_INFO: `/v3/projects/1132089273ae485cbb876aaf9cd3a507/users/b74a8981db5742d28cf099ec29b6f911/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:10.389516 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5685c-1d10-46d5-a6bd-856ec45af557 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1132089273ae485cbb876aaf9cd3a507, user_id=b74a8981db5742d28cf099ec29b6f911, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:10.392786 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5685c-1d10-46d5-a6bd-856ec45af557 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:10.429794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 836/1660] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:10 2022] PUT /identity/v3/projects/1132089273ae485cbb876aaf9cd3a507/users/b74a8981db5742d28cf099ec29b6f911/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:26:10.441229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddd509-2f22-4005-bf15-777111308b0e None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:10.441809 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddd509-2f22-4005-bf15-777111308b0e None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:10.442046 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddd509-2f22-4005-bf15-777111308b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:10.494237 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41d6cd-c74d-4cf6-ac1c-f123550d8b6e None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:10.494870 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41d6cd-c74d-4cf6-ac1c-f123550d8b6e None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:10.495366 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41d6cd-c74d-4cf6-ac1c-f123550d8b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:10.535303 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-61ddd509-2f22-4005-bf15-777111308b0e None None] MFA Rules not processed for user `b74a8981db5742d28cf099ec29b6f911`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:10.541596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-61ddd509-2f22-4005-bf15-777111308b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:10.581814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-9f41d6cd-c74d-4cf6-ac1c-f123550d8b6e None None] MFA Rules not processed for user `5ce40a75512a4ff9971552ffc54136c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:10.584610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9f41d6cd-c74d-4cf6-ac1c-f123550d8b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:10.771160 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:10.786844 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 825/1661] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:10.798530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddd509-2f22-4005-bf15-777111308b0e None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:10.810841 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:10.830487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 837/1662] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:26:10.842382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fc082-bd49-47b7-b89b-6053da9f950a None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:10.842773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fc082-bd49-47b7-b89b-6053da9f950a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:10.843148 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fc082-bd49-47b7-b89b-6053da9f950a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:10.879835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-3c6fc082-bd49-47b7-b89b-6053da9f950a None None] MFA Rules not processed for user `5ce40a75512a4ff9971552ffc54136c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:10.882581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3c6fc082-bd49-47b7-b89b-6053da9f950a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:10.912270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 838/1663] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:10.924247 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fc082-bd49-47b7-b89b-6053da9f950a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:10.932735 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3c6fc082-bd49-47b7-b89b-6053da9f950a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:10.958498 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddd509-2f22-4005-bf15-777111308b0e None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:10.966443 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902c61d-02d0-4333-846c-a7c720a6f4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:10.967402 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902c61d-02d0-4333-846c-a7c720a6f4bd admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:10.967846 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902c61d-02d0-4333-846c-a7c720a6f4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:10.968305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902c61d-02d0-4333-846c-a7c720a6f4bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:10.969103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902c61d-02d0-4333-846c-a7c720a6f4bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:10.971608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902c61d-02d0-4333-846c-a7c720a6f4bd admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:11.082800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 826/1664] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:10 2022] POST /identity/v3/projects => generated 387 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:11.087331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902c61d-02d0-4333-846c-a7c720a6f4bd admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:11.241440 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fc082-bd49-47b7-b89b-6053da9f950a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:11.380462 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902c61d-02d0-4333-846c-a7c720a6f4bd admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:11.388681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881c15e-d9a5-4b66-b526-21dc8f409aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:11.389390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881c15e-d9a5-4b66-b526-21dc8f409aba admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:11.389677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881c15e-d9a5-4b66-b526-21dc8f409aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:11.389961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881c15e-d9a5-4b66-b526-21dc8f409aba admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:11.390622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881c15e-d9a5-4b66-b526-21dc8f409aba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:11.392953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881c15e-d9a5-4b66-b526-21dc8f409aba admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:11.454457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 827/1665] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:11 2022] POST /identity/v3/users => generated 300 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:11.459933 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881c15e-d9a5-4b66-b526-21dc8f409aba admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:11.533939 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881c15e-d9a5-4b66-b526-21dc8f409aba admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:11.561876 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f007e-db28-49d0-a487-adbbe630bcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:11.562521 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f007e-db28-49d0-a487-adbbe630bcf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:11.562800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f007e-db28-49d0-a487-adbbe630bcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:11.563076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f007e-db28-49d0-a487-adbbe630bcf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:11.563745 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f007e-db28-49d0-a487-adbbe630bcf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:11.565509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f007e-db28-49d0-a487-adbbe630bcf2 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:11.602301 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 828/1666] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:11.614691 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f007e-db28-49d0-a487-adbbe630bcf2 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:11.623912 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74564d65-639f-4a0f-bac0-e55ac0a1b27c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:11.626066 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74564d65-639f-4a0f-bac0-e55ac0a1b27c service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:11.627746 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74564d65-639f-4a0f-bac0-e55ac0a1b27c service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:11.630201 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74564d65-639f-4a0f-bac0-e55ac0a1b27c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:11.647798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f007e-db28-49d0-a487-adbbe630bcf2 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:11.662382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f5c8f-e9f7-433b-805e-426952c0c8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:11.663668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f5c8f-e9f7-433b-805e-426952c0c8a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:11.668690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f5c8f-e9f7-433b-805e-426952c0c8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:11.669602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f5c8f-e9f7-433b-805e-426952c0c8a1 admin admin] PATH_INFO: `/v3/projects/db1aa4c58c6b42c8b84d258935d3f494/users/8cefa2eb51a742bab75679b4789072b0/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:11.717435 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f5c8f-e9f7-433b-805e-426952c0c8a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db1aa4c58c6b42c8b84d258935d3f494, user_id=8cefa2eb51a742bab75679b4789072b0, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:11.728845 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f5c8f-e9f7-433b-805e-426952c0c8a1 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:11.787450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 829/1667] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:11 2022] PUT /identity/v3/projects/db1aa4c58c6b42c8b84d258935d3f494/users/8cefa2eb51a742bab75679b4789072b0/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:11.793929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f5c8f-e9f7-433b-805e-426952c0c8a1 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:12.026307 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74564d65-639f-4a0f-bac0-e55ac0a1b27c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:12.034271 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74564d65-639f-4a0f-bac0-e55ac0a1b27c service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:12.054947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:12.117419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 839/1668] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 1194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:12.123551 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996d276-0782-4678-8e47-2e31e0adc777 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:12.125051 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996d276-0782-4678-8e47-2e31e0adc777 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:12.125989 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996d276-0782-4678-8e47-2e31e0adc777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:12.143739 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f5c8f-e9f7-433b-805e-426952c0c8a1 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:12.163130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb623e1-a9cf-45a5-aa6f-a77ad057112e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:12.171911 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb623e1-a9cf-45a5-aa6f-a77ad057112e admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:12.173233 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb623e1-a9cf-45a5-aa6f-a77ad057112e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:12.173456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb623e1-a9cf-45a5-aa6f-a77ad057112e admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:12.174710 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb623e1-a9cf-45a5-aa6f-a77ad057112e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:12.183941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb623e1-a9cf-45a5-aa6f-a77ad057112e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:12.222728 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 830/1669] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:12.227266 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb623e1-a9cf-45a5-aa6f-a77ad057112e admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:12.234948 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-0996d276-0782-4678-8e47-2e31e0adc777 None None] MFA Rules not processed for user `f643fd4ee6bc4ccfae45f3b8523ae46a`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:12.238041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0996d276-0782-4678-8e47-2e31e0adc777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:12.269177 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:12.287489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb623e1-a9cf-45a5-aa6f-a77ad057112e admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:12.310692 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22419b-357a-4297-aada-60222c2eab4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:12.314600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22419b-357a-4297-aada-60222c2eab4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:12.316152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22419b-357a-4297-aada-60222c2eab4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:12.316533 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22419b-357a-4297-aada-60222c2eab4f admin admin] PATH_INFO: `/v3/projects/db1aa4c58c6b42c8b84d258935d3f494/users/8cefa2eb51a742bab75679b4789072b0/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:12.343058 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22419b-357a-4297-aada-60222c2eab4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=db1aa4c58c6b42c8b84d258935d3f494, user_id=8cefa2eb51a742bab75679b4789072b0, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:12.350125 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22419b-357a-4297-aada-60222c2eab4f admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:12.452448 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 831/1670] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:12 2022] PUT /identity/v3/projects/db1aa4c58c6b42c8b84d258935d3f494/users/8cefa2eb51a742bab75679b4789072b0/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:26:12.471943 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbdf67-bc1b-413f-b6aa-5c9978820e17 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:12.472467 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbdf67-bc1b-413f-b6aa-5c9978820e17 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:12.472906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbdf67-bc1b-413f-b6aa-5c9978820e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:12.583121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-a4cbdf67-bc1b-413f-b6aa-5c9978820e17 None None] MFA Rules not processed for user `8cefa2eb51a742bab75679b4789072b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:12.586282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-a4cbdf67-bc1b-413f-b6aa-5c9978820e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:12.660695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 840/1671] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 542 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:12.677138 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5f4df-92a9-4c25-8925-8e3d183c376a None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:12.678052 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5f4df-92a9-4c25-8925-8e3d183c376a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:12.680858 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5f4df-92a9-4c25-8925-8e3d183c376a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:12.789248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-68b5f4df-92a9-4c25-8925-8e3d183c376a None None] MFA Rules not processed for user `f643fd4ee6bc4ccfae45f3b8523ae46a`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:12.795290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-68b5f4df-92a9-4c25-8925-8e3d183c376a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:12.996880 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 832/1672] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 530 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:13.184293 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbdf67-bc1b-413f-b6aa-5c9978820e17 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:13.192684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-a4cbdf67-bc1b-413f-b6aa-5c9978820e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:13.221995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 841/1673] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 546 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:13.428231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbdf67-bc1b-413f-b6aa-5c9978820e17 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:13.434031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e6f3a-0c92-47ac-8fd2-ffce44096337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:13.434617 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e6f3a-0c92-47ac-8fd2-ffce44096337 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:13.435197 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e6f3a-0c92-47ac-8fd2-ffce44096337 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:13.436771 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e6f3a-0c92-47ac-8fd2-ffce44096337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:13.474983 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e6f3a-0c92-47ac-8fd2-ffce44096337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:13.484147 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e6f3a-0c92-47ac-8fd2-ffce44096337 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:13.544936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 833/1674] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:13.744960 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5f4df-92a9-4c25-8925-8e3d183c376a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:13.855427 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e6f3a-0c92-47ac-8fd2-ffce44096337 service cinder] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:13.856975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d75e6f3a-0c92-47ac-8fd2-ffce44096337 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:14.105834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5f4df-92a9-4c25-8925-8e3d183c376a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:14.118916 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d280a-6830-4099-8087-5d30221cef5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:14.121390 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d280a-6830-4099-8087-5d30221cef5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:14.121841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d280a-6830-4099-8087-5d30221cef5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:14.122366 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d280a-6830-4099-8087-5d30221cef5c admin admin] PATH_INFO: `/v3/users/b74a8981db5742d28cf099ec29b6f911` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:14.129702 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d280a-6830-4099-8087-5d30221cef5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b74a8981db5742d28cf099ec29b6f911)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:14.130909 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: WARNING py.warnings [None req-b72d280a-6830-4099-8087-5d30221cef5c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 19:26:14.130909 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: warnings.warn(msg) Aug 07 19:26:14.130909 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: Aug 07 19:26:14.132504 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d280a-6830-4099-8087-5d30221cef5c admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:14.199671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:14.299290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e6f3a-0c92-47ac-8fd2-ffce44096337 service cinder] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:14.300086 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-b72d280a-6830-4099-8087-5d30221cef5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b74a8981db5742d28cf099ec29b6f911'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:14.322580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f52e3a-6bb6-4fc8-bb6f-2bb6f7e73226 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:14.323236 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f52e3a-6bb6-4fc8-bb6f-2bb6f7e73226 service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:14.323527 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f52e3a-6bb6-4fc8-bb6f-2bb6f7e73226 service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:14.323802 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f52e3a-6bb6-4fc8-bb6f-2bb6f7e73226 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:14.326958 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-b72d280a-6830-4099-8087-5d30221cef5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b74a8981db5742d28cf099ec29b6f911'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:14.330499 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:14.345240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f52e3a-6bb6-4fc8-bb6f-2bb6f7e73226 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:14.346775 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f52e3a-6bb6-4fc8-bb6f-2bb6f7e73226 service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:14.368505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 834/1675] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:14.370182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61bda2-213e-4a94-b0dc-0cd5fd0c4b66 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:14.371939 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61bda2-213e-4a94-b0dc-0cd5fd0c4b66 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:14.373644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61bda2-213e-4a94-b0dc-0cd5fd0c4b66 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:14.380138 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 835/1676] 10.222.0.17 () {54 vars in 912 bytes} [Sun Aug 7 19:26:14 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:26:14.383806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-b72d280a-6830-4099-8087-5d30221cef5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b74a8981db5742d28cf099ec29b6f911'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:14.389212 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715e229-6eb3-4b06-9ade-abf6fd5e0986 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:14.389870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715e229-6eb3-4b06-9ade-abf6fd5e0986 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:14.392378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-b72d280a-6830-4099-8087-5d30221cef5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b74a8981db5742d28cf099ec29b6f911'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:14.392941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715e229-6eb3-4b06-9ade-abf6fd5e0986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:14.435451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-b72d280a-6830-4099-8087-5d30221cef5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b74a8981db5742d28cf099ec29b6f911'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:14.450144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 842/1677] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:13 2022] DELETE /identity/v3/users/b74a8981db5742d28cf099ec29b6f911 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:14.457624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2637fb2-35e6-4c91-a229-b342b1312e01 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:14.457957 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2637fb2-35e6-4c91-a229-b342b1312e01 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:14.458196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2637fb2-35e6-4c91-a229-b342b1312e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:14.527805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-e2637fb2-35e6-4c91-a229-b342b1312e01 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:14.529821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e2637fb2-35e6-4c91-a229-b342b1312e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:14.537854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-6715e229-6eb3-4b06-9ade-abf6fd5e0986 None None] MFA Rules not processed for user `08229e9400b14e8a9cc8e8c32dc48285`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:14.539579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6715e229-6eb3-4b06-9ade-abf6fd5e0986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:14.714352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 843/1678] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:14.719460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c07f30-b085-4ea8-ac95-b2b264cbc4f8 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:14.720437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c07f30-b085-4ea8-ac95-b2b264cbc4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:14.721333 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 836/1679] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Aug 7 19:26:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:14.721803 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c07f30-b085-4ea8-ac95-b2b264cbc4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:14.779179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715e229-6eb3-4b06-9ade-abf6fd5e0986 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:14.781320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6715e229-6eb3-4b06-9ade-abf6fd5e0986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:14.800164 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-a3c07f30-b085-4ea8-ac95-b2b264cbc4f8 None None] MFA Rules not processed for user `6732487041c54918b1164b1cb72bb07e`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:14.801517 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715e229-6eb3-4b06-9ade-abf6fd5e0986 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:14.801767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a3c07f30-b085-4ea8-ac95-b2b264cbc4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:14.817313 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a81f49-8cf5-445e-a355-5e8c3e017f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:14.818219 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a81f49-8cf5-445e-a355-5e8c3e017f9c service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:14.818670 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a81f49-8cf5-445e-a355-5e8c3e017f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:14.819086 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a81f49-8cf5-445e-a355-5e8c3e017f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:14.840476 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a81f49-8cf5-445e-a355-5e8c3e017f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:14.842582 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a81f49-8cf5-445e-a355-5e8c3e017f9c service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:14.875558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 837/1680] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:14.878762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a81f49-8cf5-445e-a355-5e8c3e017f9c service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:14.905427 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a81f49-8cf5-445e-a355-5e8c3e017f9c service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:14.913927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671740a-503d-49bc-8ec5-1a6472ff57ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:14.914868 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671740a-503d-49bc-8ec5-1a6472ff57ea service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:14.915132 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671740a-503d-49bc-8ec5-1a6472ff57ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:14.917141 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671740a-503d-49bc-8ec5-1a6472ff57ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:14.944728 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671740a-503d-49bc-8ec5-1a6472ff57ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:14.946450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671740a-503d-49bc-8ec5-1a6472ff57ea service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:14.964970 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 844/1681] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:14.972265 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 838/1682] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:15.416632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c07f30-b085-4ea8-ac95-b2b264cbc4f8 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:15.418173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a3c07f30-b085-4ea8-ac95-b2b264cbc4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:15.446583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c07f30-b085-4ea8-ac95-b2b264cbc4f8 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:15.461481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b96d0cc-f1be-4214-a764-21a714484dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:15.462469 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96d0cc-f1be-4214-a764-21a714484dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:15.462749 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96d0cc-f1be-4214-a764-21a714484dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:15.463119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b96d0cc-f1be-4214-a764-21a714484dcb admin admin] PATH_INFO: `/v3/users/a51a34944fef4fb284d4d835ec03fb61` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:15.466213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96d0cc-f1be-4214-a764-21a714484dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a51a34944fef4fb284d4d835ec03fb61)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:15.471096 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b96d0cc-f1be-4214-a764-21a714484dcb admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:15.503361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7b6ab-2955-4419-b9ee-025231990d2e None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:15.503361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7b6ab-2955-4419-b9ee-025231990d2e None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:15.503575 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7b6ab-2955-4419-b9ee-025231990d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:15.579008 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-0bb7b6ab-2955-4419-b9ee-025231990d2e None None] MFA Rules not processed for user `be3e2afc428a45e2b993c39e3d53cd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:15.582895 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-0bb7b6ab-2955-4419-b9ee-025231990d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:15.631777 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3b96d0cc-f1be-4214-a764-21a714484dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a51a34944fef4fb284d4d835ec03fb61'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:15.653118 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3b96d0cc-f1be-4214-a764-21a714484dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a51a34944fef4fb284d4d835ec03fb61'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:15.660503 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:15.701448 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3b96d0cc-f1be-4214-a764-21a714484dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a51a34944fef4fb284d4d835ec03fb61'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:15.703078 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3b96d0cc-f1be-4214-a764-21a714484dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a51a34944fef4fb284d4d835ec03fb61'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:15.724529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3b96d0cc-f1be-4214-a764-21a714484dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a51a34944fef4fb284d4d835ec03fb61'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:15.748153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 845/1683] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:15 2022] DELETE /identity/v3/users/a51a34944fef4fb284d4d835ec03fb61 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:26:15.748153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810a52c-f201-40a9-81dd-2df20c66f4cd None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:15.749299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810a52c-f201-40a9-81dd-2df20c66f4cd None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:15.749722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810a52c-f201-40a9-81dd-2df20c66f4cd None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:15.755427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 846/1684] 10.222.0.17 () {54 vars in 912 bytes} [Sun Aug 7 19:26:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:26:15.759145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56bca4-4256-45ba-ac18-4df0e5bf5a36 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:15.760750 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56bca4-4256-45ba-ac18-4df0e5bf5a36 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:15.761363 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56bca4-4256-45ba-ac18-4df0e5bf5a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:15.806146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 839/1685] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:26:15.809038 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c3ab7-137e-44e9-b194-de9c7daf1396 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:15.810716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c3ab7-137e-44e9-b194-de9c7daf1396 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:15.811370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c3ab7-137e-44e9-b194-de9c7daf1396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:15.835523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-3f56bca4-4256-45ba-ac18-4df0e5bf5a36 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:15.841353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3f56bca4-4256-45ba-ac18-4df0e5bf5a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:15.874966 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-052c3ab7-137e-44e9-b194-de9c7daf1396 None None] MFA Rules not processed for user `08229e9400b14e8a9cc8e8c32dc48285`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:15.879454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-052c3ab7-137e-44e9-b194-de9c7daf1396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:16.008729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 847/1686] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:16.033718 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56bca4-4256-45ba-ac18-4df0e5bf5a36 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:16.037653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3f56bca4-4256-45ba-ac18-4df0e5bf5a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:16.159274 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 840/1687] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Aug 7 19:26:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:16.165731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56bca4-4256-45ba-ac18-4df0e5bf5a36 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:16.170288 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6738cf-e9b4-43a9-877f-4b85d9188b18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:16.171186 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6738cf-e9b4-43a9-877f-4b85d9188b18 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:16.171535 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6738cf-e9b4-43a9-877f-4b85d9188b18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:16.171876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6738cf-e9b4-43a9-877f-4b85d9188b18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:16.175192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c3ab7-137e-44e9-b194-de9c7daf1396 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:16.186843 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-052c3ab7-137e-44e9-b194-de9c7daf1396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:16.186953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6738cf-e9b4-43a9-877f-4b85d9188b18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:16.189375 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6738cf-e9b4-43a9-877f-4b85d9188b18 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:16.201829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 848/1688] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:16.203430 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c24231-fc8a-4116-a20c-3f6e1ffdc430 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:16.203790 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c24231-fc8a-4116-a20c-3f6e1ffdc430 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:16.204161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c24231-fc8a-4116-a20c-3f6e1ffdc430 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:16.206161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 849/1689] 10.222.0.17 () {54 vars in 961 bytes} [Sun Aug 7 19:26:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:26:16.207036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c24231-fc8a-4116-a20c-3f6e1ffdc430 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:16.209153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-22c24231-fc8a-4116-a20c-3f6e1ffdc430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:16.237917 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:16.249704 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c3ab7-137e-44e9-b194-de9c7daf1396 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:16.256467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c24231-fc8a-4116-a20c-3f6e1ffdc430 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:16.262256 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:16.263440 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:16.263775 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:16.264108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a admin admin] PATH_INFO: `/v3/projects/1132089273ae485cbb876aaf9cd3a507` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:16.270920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec1184-b4bb-45b1-b2ab-3c92b9af9fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:16.271636 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec1184-b4bb-45b1-b2ab-3c92b9af9fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:16.271945 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec1184-b4bb-45b1-b2ab-3c92b9af9fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:16.272280 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec1184-b4bb-45b1-b2ab-3c92b9af9fa5 admin admin] PATH_INFO: `/v3/users/ddb92150d9be4326b613600de6aeb66b` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:16.274848 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec1184-b4bb-45b1-b2ab-3c92b9af9fa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddb92150d9be4326b613600de6aeb66b)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:16.283767 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec1184-b4bb-45b1-b2ab-3c92b9af9fa5 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:16.291283 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1132089273ae485cbb876aaf9cd3a507)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:16.294922 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:16.387269 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1132089273ae485cbb876aaf9cd3a507', 'request_id': 'req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:16.399613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1132089273ae485cbb876aaf9cd3a507', 'request_id': 'req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:16.412476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1132089273ae485cbb876aaf9cd3a507', 'request_id': 'req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:16.435513 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-27ec1184-b4bb-45b1-b2ab-3c92b9af9fa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddb92150d9be4326b613600de6aeb66b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:16.435965 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a admin admin] The token cache is being invalidate because project 1132089273ae485cbb876aaf9cd3a507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:16.436484 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:16.438568 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:16.440990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 850/1690] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:16 2022] DELETE /identity/v3/projects/1132089273ae485cbb876aaf9cd3a507 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:16.442248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:16.444855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:16.462009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-27ec1184-b4bb-45b1-b2ab-3c92b9af9fa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddb92150d9be4326b613600de6aeb66b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:16.466770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:16.486335 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-27ec1184-b4bb-45b1-b2ab-3c92b9af9fa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddb92150d9be4326b613600de6aeb66b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:16.492233 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-27ec1184-b4bb-45b1-b2ab-3c92b9af9fa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddb92150d9be4326b613600de6aeb66b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:16.494738 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-27ec1184-b4bb-45b1-b2ab-3c92b9af9fa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddb92150d9be4326b613600de6aeb66b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:16.547508 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 841/1691] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:16 2022] DELETE /identity/v3/users/ddb92150d9be4326b613600de6aeb66b => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:16.550486 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a27de-d1a6-4696-9862-c596e55194f1 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:16.553079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a27de-d1a6-4696-9862-c596e55194f1 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:16.553620 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a27de-d1a6-4696-9862-c596e55194f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:16.635146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-131a27de-d1a6-4696-9862-c596e55194f1 None None] MFA Rules not processed for user `143c30b4201645279564e72ab3288adc`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:16.638972 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-131a27de-d1a6-4696-9862-c596e55194f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:16.653501 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945488d-7081-4fb0-9f1d-44b9a27c6f2a admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:16.775127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c624088-119b-4e48-9390-b5edfe1c170a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:16.777094 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c624088-119b-4e48-9390-b5edfe1c170a service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:16.777399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c624088-119b-4e48-9390-b5edfe1c170a service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:16.777675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c624088-119b-4e48-9390-b5edfe1c170a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:16.780908 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8c624088-119b-4e48-9390-b5edfe1c170a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:16.784850 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 842/1692] 10.222.0.17 () {58 vars in 1058 bytes} [Sun Aug 7 19:26:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:16.785859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a27de-d1a6-4696-9862-c596e55194f1 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:16.787398 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-131a27de-d1a6-4696-9862-c596e55194f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:16.794756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c624088-119b-4e48-9390-b5edfe1c170a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:16.796667 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c624088-119b-4e48-9390-b5edfe1c170a service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:16.840208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 851/1693] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:16.842260 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c624088-119b-4e48-9390-b5edfe1c170a service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:16.850502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8c624088-119b-4e48-9390-b5edfe1c170a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:17.026348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a27de-d1a6-4696-9862-c596e55194f1 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:17.036505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600604ac-0f6b-40cd-93d3-51f8338cc91e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:17.037426 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600604ac-0f6b-40cd-93d3-51f8338cc91e service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:17.037794 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600604ac-0f6b-40cd-93d3-51f8338cc91e service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:17.038131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600604ac-0f6b-40cd-93d3-51f8338cc91e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:17.040287 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-600604ac-0f6b-40cd-93d3-51f8338cc91e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:17.059401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:17.070248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c624088-119b-4e48-9390-b5edfe1c170a service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:17.082082 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39106a-7494-4fc9-a45e-8854cb9846af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:17.082858 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39106a-7494-4fc9-a45e-8854cb9846af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:17.083230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39106a-7494-4fc9-a45e-8854cb9846af admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:17.083615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39106a-7494-4fc9-a45e-8854cb9846af admin admin] PATH_INFO: `/v3/users/8cefa2eb51a742bab75679b4789072b0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:17.085034 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39106a-7494-4fc9-a45e-8854cb9846af admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cefa2eb51a742bab75679b4789072b0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:17.087230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39106a-7494-4fc9-a45e-8854cb9846af admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:17.193478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600604ac-0f6b-40cd-93d3-51f8338cc91e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:17.196916 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600604ac-0f6b-40cd-93d3-51f8338cc91e service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:17.205133 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3f39106a-7494-4fc9-a45e-8854cb9846af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cefa2eb51a742bab75679b4789072b0'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:17.214963 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3f39106a-7494-4fc9-a45e-8854cb9846af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cefa2eb51a742bab75679b4789072b0'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:17.222798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:17.238593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 843/1694] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:17.241827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8660467-ffcb-4bf8-b662-166d5f6cbdc3 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:17.242374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8660467-ffcb-4bf8-b662-166d5f6cbdc3 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:17.243058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3f39106a-7494-4fc9-a45e-8854cb9846af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cefa2eb51a742bab75679b4789072b0'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:17.244236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3f39106a-7494-4fc9-a45e-8854cb9846af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cefa2eb51a742bab75679b4789072b0'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:17.244694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8660467-ffcb-4bf8-b662-166d5f6cbdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:17.259351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3f39106a-7494-4fc9-a45e-8854cb9846af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cefa2eb51a742bab75679b4789072b0'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:17.271207 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 852/1695] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:16 2022] DELETE /identity/v3/users/8cefa2eb51a742bab75679b4789072b0 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:17.276055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39106a-7494-4fc9-a45e-8854cb9846af admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:17.277816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3f39106a-7494-4fc9-a45e-8854cb9846af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:17.323443 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-d8660467-ffcb-4bf8-b662-166d5f6cbdc3 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:17.324815 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d8660467-ffcb-4bf8-b662-166d5f6cbdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:17.479093 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 844/1696] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:17.482147 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8660467-ffcb-4bf8-b662-166d5f6cbdc3 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:17.484453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d8660467-ffcb-4bf8-b662-166d5f6cbdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:17.495060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39106a-7494-4fc9-a45e-8854cb9846af admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:17.501179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8660467-ffcb-4bf8-b662-166d5f6cbdc3 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:17.509417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cb4b6-d798-487a-9884-54d26d01d610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:17.509833 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cb4b6-d798-487a-9884-54d26d01d610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:17.510212 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cb4b6-d798-487a-9884-54d26d01d610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:17.510973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cb4b6-d798-487a-9884-54d26d01d610 admin admin] PATH_INFO: `/v3/projects/493acaa0d6944f94a0dbae8b55b33b88` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:17.512942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1cb4b6-d798-487a-9884-54d26d01d610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=493acaa0d6944f94a0dbae8b55b33b88)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:17.515581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: WARNING py.warnings [None req-3b1cb4b6-d798-487a-9884-54d26d01d610 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 19:26:17.515581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: warnings.warn(msg) Aug 07 19:26:17.515581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: Aug 07 19:26:17.517600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1cb4b6-d798-487a-9884-54d26d01d610 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:17.521097 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6f9ff-0041-4cdf-a735-9bce07c6bbb3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '143c30b4201645279564e72ab3288adc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:17.526584 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6f9ff-0041-4cdf-a735-9bce07c6bbb3 service placement] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:17.535705 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6f9ff-0041-4cdf-a735-9bce07c6bbb3 service placement] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:17.536082 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa6f9ff-0041-4cdf-a735-9bce07c6bbb3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:17.545958 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-efa6f9ff-0041-4cdf-a735-9bce07c6bbb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:17.598482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa6f9ff-0041-4cdf-a735-9bce07c6bbb3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:17.601157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa6f9ff-0041-4cdf-a735-9bce07c6bbb3 service placement] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:17.667250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-3b1cb4b6-d798-487a-9884-54d26d01d610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '493acaa0d6944f94a0dbae8b55b33b88', 'request_id': 'req-3b1cb4b6-d798-487a-9884-54d26d01d610'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:17.674137 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 853/1697] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:17.675494 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6f9ff-0041-4cdf-a735-9bce07c6bbb3 service placement] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:17.679732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-efa6f9ff-0041-4cdf-a735-9bce07c6bbb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:17.721447 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-3b1cb4b6-d798-487a-9884-54d26d01d610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '493acaa0d6944f94a0dbae8b55b33b88', 'request_id': 'req-3b1cb4b6-d798-487a-9884-54d26d01d610'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:17.746235 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-3b1cb4b6-d798-487a-9884-54d26d01d610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '493acaa0d6944f94a0dbae8b55b33b88', 'request_id': 'req-3b1cb4b6-d798-487a-9884-54d26d01d610'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:17.779714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-3b1cb4b6-d798-487a-9884-54d26d01d610 admin admin] The token cache is being invalidate because project 493acaa0d6944f94a0dbae8b55b33b88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:17.782200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-3b1cb4b6-d798-487a-9884-54d26d01d610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:17.783583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-3b1cb4b6-d798-487a-9884-54d26d01d610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:17.787806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 845/1698] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:17 2022] DELETE /identity/v3/projects/493acaa0d6944f94a0dbae8b55b33b88 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:17.792415 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cb4b6-d798-487a-9884-54d26d01d610 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:17.796922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-3b1cb4b6-d798-487a-9884-54d26d01d610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:17.896584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:17.897267 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa6f9ff-0041-4cdf-a735-9bce07c6bbb3 service placement] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:18.005275 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cb4b6-d798-487a-9884-54d26d01d610 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:18.019874 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6adeb-d588-4cca-94a3-0ddb82d66264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:18.020763 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6adeb-d588-4cca-94a3-0ddb82d66264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:18.021171 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6adeb-d588-4cca-94a3-0ddb82d66264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:18.021617 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b6adeb-d588-4cca-94a3-0ddb82d66264 admin admin] PATH_INFO: `/v3/users/50fb339b2fa74f9192730d34df80a1a1` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:18.028964 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6adeb-d588-4cca-94a3-0ddb82d66264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50fb339b2fa74f9192730d34df80a1a1)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:18.039835 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b6adeb-d588-4cca-94a3-0ddb82d66264 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:18.158120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-39b6adeb-d588-4cca-94a3-0ddb82d66264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50fb339b2fa74f9192730d34df80a1a1'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:18.176817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-39b6adeb-d588-4cca-94a3-0ddb82d66264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50fb339b2fa74f9192730d34df80a1a1'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:18.188404 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-39b6adeb-d588-4cca-94a3-0ddb82d66264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50fb339b2fa74f9192730d34df80a1a1'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:18.190267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-39b6adeb-d588-4cca-94a3-0ddb82d66264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50fb339b2fa74f9192730d34df80a1a1'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:18.191102 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-39b6adeb-d588-4cca-94a3-0ddb82d66264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50fb339b2fa74f9192730d34df80a1a1'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:18.203571 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960ceb1-0c72-42b6-b5c2-04f18e43e2cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:18.218414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960ceb1-0c72-42b6-b5c2-04f18e43e2cd service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:18.225253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960ceb1-0c72-42b6-b5c2-04f18e43e2cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:18.225968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960ceb1-0c72-42b6-b5c2-04f18e43e2cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:18.227956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 846/1699] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:17 2022] DELETE /identity/v3/users/50fb339b2fa74f9192730d34df80a1a1 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:18.239867 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-6960ceb1-0c72-42b6-b5c2-04f18e43e2cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:18.269394 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960ceb1-0c72-42b6-b5c2-04f18e43e2cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:18.271129 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960ceb1-0c72-42b6-b5c2-04f18e43e2cd service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:18.284995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 854/1700] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:18.466481 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6adeb-d588-4cca-94a3-0ddb82d66264 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:18.468913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-39b6adeb-d588-4cca-94a3-0ddb82d66264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:18.695421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b6adeb-d588-4cca-94a3-0ddb82d66264 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:18.712805 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7b940-9f7d-4b60-a680-cb623e381fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:18.715229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7b940-9f7d-4b60-a680-cb623e381fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:18.715642 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7b940-9f7d-4b60-a680-cb623e381fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:18.717025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7b940-9f7d-4b60-a680-cb623e381fcf admin admin] PATH_INFO: `/v3/projects/db1aa4c58c6b42c8b84d258935d3f494` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:18.722431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7b940-9f7d-4b60-a680-cb623e381fcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=db1aa4c58c6b42c8b84d258935d3f494)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:18.730382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7b940-9f7d-4b60-a680-cb623e381fcf admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:18.960505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f6f7b940-9f7d-4b60-a680-cb623e381fcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db1aa4c58c6b42c8b84d258935d3f494', 'request_id': 'req-f6f7b940-9f7d-4b60-a680-cb623e381fcf'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:18.986330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f6f7b940-9f7d-4b60-a680-cb623e381fcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db1aa4c58c6b42c8b84d258935d3f494', 'request_id': 'req-f6f7b940-9f7d-4b60-a680-cb623e381fcf'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:18.993168 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:19.010776 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f6f7b940-9f7d-4b60-a680-cb623e381fcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db1aa4c58c6b42c8b84d258935d3f494', 'request_id': 'req-f6f7b940-9f7d-4b60-a680-cb623e381fcf'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:19.041683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f6f7b940-9f7d-4b60-a680-cb623e381fcf admin admin] The token cache is being invalidate because project db1aa4c58c6b42c8b84d258935d3f494 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:19.042005 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f6f7b940-9f7d-4b60-a680-cb623e381fcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:19.045150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f6f7b940-9f7d-4b60-a680-cb623e381fcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:19.051689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 847/1701] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:18 2022] DELETE /identity/v3/projects/db1aa4c58c6b42c8b84d258935d3f494 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:19.060236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e443899-d953-4fc1-91d9-afdbd5d2c9b9 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:19.060718 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e443899-d953-4fc1-91d9-afdbd5d2c9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:19.060835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e443899-d953-4fc1-91d9-afdbd5d2c9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:19.119785 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-0e443899-d953-4fc1-91d9-afdbd5d2c9b9 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:19.139599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0e443899-d953-4fc1-91d9-afdbd5d2c9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:19.210559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:19.241980 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7b940-9f7d-4b60-a680-cb623e381fcf admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:19.243610 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f6f7b940-9f7d-4b60-a680-cb623e381fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:19.283684 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 855/1702] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:19.300408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f2543-ab53-4375-8f06-638fcb6d5ff9 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:19.300942 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f2543-ab53-4375-8f06-638fcb6d5ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:19.301297 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f2543-ab53-4375-8f06-638fcb6d5ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:19.361154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-453f2543-ab53-4375-8f06-638fcb6d5ff9 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:19.362541 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-453f2543-ab53-4375-8f06-638fcb6d5ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:19.374034 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 856/1703] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:19.397683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f2543-ab53-4375-8f06-638fcb6d5ff9 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:19.424879 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f2543-ab53-4375-8f06-638fcb6d5ff9 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:19.448024 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7f3652-eb6f-4ed7-8ef4-c8e2e99cfaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:19.448024 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f3652-eb6f-4ed7-8ef4-c8e2e99cfaed admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:19.448024 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f3652-eb6f-4ed7-8ef4-c8e2e99cfaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:19.448024 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7f3652-eb6f-4ed7-8ef4-c8e2e99cfaed admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:19.448521 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f3652-eb6f-4ed7-8ef4-c8e2e99cfaed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:26:19.448521 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f3652-eb6f-4ed7-8ef4-c8e2e99cfaed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:19.448521 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7f3652-eb6f-4ed7-8ef4-c8e2e99cfaed admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:19.485053 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 857/1704] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:26:19.487698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e8fb1-39a8-4efb-ba4a-4f1dd821187f None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:19.488067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e8fb1-39a8-4efb-ba4a-4f1dd821187f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:19.488365 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e8fb1-39a8-4efb-ba4a-4f1dd821187f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:19.519198 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7b940-9f7d-4b60-a680-cb623e381fcf admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:19.543948 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb698a-741f-4837-ae1c-d123c6b0c2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:19.544690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb698a-741f-4837-ae1c-d123c6b0c2e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:19.544983 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb698a-741f-4837-ae1c-d123c6b0c2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:19.545315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb698a-741f-4837-ae1c-d123c6b0c2e6 admin admin] PATH_INFO: `/v3/projects/babe6e4cd9a94181b5b42f4927aa98a8` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:19.549992 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb698a-741f-4837-ae1c-d123c6b0c2e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=babe6e4cd9a94181b5b42f4927aa98a8)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:19.553792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-a51e8fb1-39a8-4efb-ba4a-4f1dd821187f None None] MFA Rules not processed for user `f643fd4ee6bc4ccfae45f3b8523ae46a`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:19.560513 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb698a-741f-4837-ae1c-d123c6b0c2e6 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:19.563756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a51e8fb1-39a8-4efb-ba4a-4f1dd821187f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:19.774119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 858/1705] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:19.775412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e8fb1-39a8-4efb-ba4a-4f1dd821187f None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:19.806819 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e8fb1-39a8-4efb-ba4a-4f1dd821187f None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:19.832244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-71cb698a-741f-4837-ae1c-d123c6b0c2e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'babe6e4cd9a94181b5b42f4927aa98a8', 'request_id': 'req-71cb698a-741f-4837-ae1c-d123c6b0c2e6'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:19.855957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-71cb698a-741f-4837-ae1c-d123c6b0c2e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'babe6e4cd9a94181b5b42f4927aa98a8', 'request_id': 'req-71cb698a-741f-4837-ae1c-d123c6b0c2e6'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:19.890396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-71cb698a-741f-4837-ae1c-d123c6b0c2e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'babe6e4cd9a94181b5b42f4927aa98a8', 'request_id': 'req-71cb698a-741f-4837-ae1c-d123c6b0c2e6'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:19.952485 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf29c23-64dd-469c-acd1-fb9aa3165a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:19.953367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf29c23-64dd-469c-acd1-fb9aa3165a52 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:19.953769 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf29c23-64dd-469c-acd1-fb9aa3165a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:19.954189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf29c23-64dd-469c-acd1-fb9aa3165a52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:19.955411 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf29c23-64dd-469c-acd1-fb9aa3165a52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:19.957989 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf29c23-64dd-469c-acd1-fb9aa3165a52 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:19.958600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-71cb698a-741f-4837-ae1c-d123c6b0c2e6 admin admin] The token cache is being invalidate because project babe6e4cd9a94181b5b42f4927aa98a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:19.959402 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-71cb698a-741f-4837-ae1c-d123c6b0c2e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:19.963662 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-71cb698a-741f-4837-ae1c-d123c6b0c2e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:19.976426 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 848/1706] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:19 2022] DELETE /identity/v3/projects/babe6e4cd9a94181b5b42f4927aa98a8 => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:26:19.982345 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298291f-91b3-49db-8f8a-5953476174d1 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:19.984103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298291f-91b3-49db-8f8a-5953476174d1 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:19.988154 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298291f-91b3-49db-8f8a-5953476174d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:20.088164 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 859/1707] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:19 2022] POST /identity/v3/projects => generated 383 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:20.089912 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf29c23-64dd-469c-acd1-fb9aa3165a52 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:20.090753 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-1298291f-91b3-49db-8f8a-5953476174d1 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:20.093976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1298291f-91b3-49db-8f8a-5953476174d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:20.101649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-edf29c23-64dd-469c-acd1-fb9aa3165a52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:20.211514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:20.281999 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf29c23-64dd-469c-acd1-fb9aa3165a52 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:20.308237 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d003c3b-cc63-4497-87bd-401e495128cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:20.309128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d003c3b-cc63-4497-87bd-401e495128cc admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:20.309128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d003c3b-cc63-4497-87bd-401e495128cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:20.309453 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d003c3b-cc63-4497-87bd-401e495128cc admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:20.310127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d003c3b-cc63-4497-87bd-401e495128cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:20.314416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d003c3b-cc63-4497-87bd-401e495128cc admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:20.325347 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 849/1708] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:20.335690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4dca7-aad9-4f70-baf3-0cf37f44baa5 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:20.336228 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4dca7-aad9-4f70-baf3-0cf37f44baa5 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:20.336807 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc4dca7-aad9-4f70-baf3-0cf37f44baa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:20.387156 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-1fc4dca7-aad9-4f70-baf3-0cf37f44baa5 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:20.389475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1fc4dca7-aad9-4f70-baf3-0cf37f44baa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:20.409937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 850/1709] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:26:20.415704 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4dca7-aad9-4f70-baf3-0cf37f44baa5 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:20.431358 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc4dca7-aad9-4f70-baf3-0cf37f44baa5 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:20.445111 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 860/1710] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:20 2022] POST /identity/v3/users => generated 292 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:20.446198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d003c3b-cc63-4497-87bd-401e495128cc admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:20.455274 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2829e6-c7b9-4e23-89f4-6be920ed25c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:20.455914 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2829e6-c7b9-4e23-89f4-6be920ed25c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:20.456270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2829e6-c7b9-4e23-89f4-6be920ed25c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:20.456630 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2829e6-c7b9-4e23-89f4-6be920ed25c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:20.457362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2829e6-c7b9-4e23-89f4-6be920ed25c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:26:20.457820 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2829e6-c7b9-4e23-89f4-6be920ed25c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:20.465593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2829e6-c7b9-4e23-89f4-6be920ed25c5 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:20.470475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:20.484769 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d003c3b-cc63-4497-87bd-401e495128cc admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:20.499659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e851cc-7e80-4022-b595-747d637b772d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:20.500396 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e851cc-7e80-4022-b595-747d637b772d admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:20.500671 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e851cc-7e80-4022-b595-747d637b772d admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:20.500945 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e851cc-7e80-4022-b595-747d637b772d admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:20.501690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e851cc-7e80-4022-b595-747d637b772d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:20.503408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e851cc-7e80-4022-b595-747d637b772d admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:20.517531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 851/1711] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:26:20.525761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2829e6-c7b9-4e23-89f4-6be920ed25c5 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:20.539717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2829e6-c7b9-4e23-89f4-6be920ed25c5 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:20.541586 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 861/1712] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:20.546391 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e851cc-7e80-4022-b595-747d637b772d admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:20.563187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e851cc-7e80-4022-b595-747d637b772d admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:20.564276 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3499889-2cee-4247-8aff-6d213461bb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:20.569866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3499889-2cee-4247-8aff-6d213461bb74 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:20.569866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3499889-2cee-4247-8aff-6d213461bb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:20.572097 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3499889-2cee-4247-8aff-6d213461bb74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:20.572332 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3499889-2cee-4247-8aff-6d213461bb74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:20.574778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30111a83-a4f3-4212-bb31-ea12096835ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:20.575015 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30111a83-a4f3-4212-bb31-ea12096835ba admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:20.575193 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30111a83-a4f3-4212-bb31-ea12096835ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:20.575469 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30111a83-a4f3-4212-bb31-ea12096835ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:20.577686 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30111a83-a4f3-4212-bb31-ea12096835ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:20.578740 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3499889-2cee-4247-8aff-6d213461bb74 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:20.584817 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30111a83-a4f3-4212-bb31-ea12096835ba admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:20.601674 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 862/1713] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:26:20.606563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30111a83-a4f3-4212-bb31-ea12096835ba admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:20.628906 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30111a83-a4f3-4212-bb31-ea12096835ba admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:20.637151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01030af9-8074-4518-9502-75c5ff3a6e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:20.637963 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01030af9-8074-4518-9502-75c5ff3a6e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:20.638323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01030af9-8074-4518-9502-75c5ff3a6e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:20.638688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01030af9-8074-4518-9502-75c5ff3a6e48 admin admin] PATH_INFO: `/v3/projects/d6d71958f4f042618dce65173e4a7180/users/ee32aee54d884eccaf17c81853f214f3/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:20.676816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 852/1714] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:20 2022] POST /identity/v3/projects => generated 369 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:20.683656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3499889-2cee-4247-8aff-6d213461bb74 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:20.713634 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01030af9-8074-4518-9502-75c5ff3a6e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6d71958f4f042618dce65173e4a7180, user_id=ee32aee54d884eccaf17c81853f214f3, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:20.726377 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01030af9-8074-4518-9502-75c5ff3a6e48 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:20.787247 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 863/1715] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:20 2022] PUT /identity/v3/projects/d6d71958f4f042618dce65173e4a7180/users/ee32aee54d884eccaf17c81853f214f3/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:20.800453 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a4aa-2984-47a3-9809-ac66de796c7e None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:20.801327 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a4aa-2984-47a3-9809-ac66de796c7e None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:20.802832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702a4aa-2984-47a3-9809-ac66de796c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:20.886620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-7702a4aa-2984-47a3-9809-ac66de796c7e None None] MFA Rules not processed for user `ee32aee54d884eccaf17c81853f214f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:20.888472 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-7702a4aa-2984-47a3-9809-ac66de796c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:20.919519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3499889-2cee-4247-8aff-6d213461bb74 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:21.151877 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbead2-134d-4827-98c1-37d336cc6f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:21.159789 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbead2-134d-4827-98c1-37d336cc6f89 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:21.165601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbead2-134d-4827-98c1-37d336cc6f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:21.166211 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cbead2-134d-4827-98c1-37d336cc6f89 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:21.174511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbead2-134d-4827-98c1-37d336cc6f89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:21.194961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cbead2-134d-4827-98c1-37d336cc6f89 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:21.254347 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 864/1716] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:21.263361 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702a4aa-2984-47a3-9809-ac66de796c7e None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:21.394328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 853/1717] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:20 2022] POST /identity/v3/users => generated 285 bytes in 713 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:21.398600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbead2-134d-4827-98c1-37d336cc6f89 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:21.403978 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702a4aa-2984-47a3-9809-ac66de796c7e None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:21.413071 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-35cbead2-134d-4827-98c1-37d336cc6f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:21.456052 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fe8a0-cb9c-461c-a0cb-e4abda95e050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:21.469507 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fe8a0-cb9c-461c-a0cb-e4abda95e050 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:21.473542 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fe8a0-cb9c-461c-a0cb-e4abda95e050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:21.480887 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fe8a0-cb9c-461c-a0cb-e4abda95e050 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:21.485978 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7fe8a0-cb9c-461c-a0cb-e4abda95e050 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:21.492461 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7fe8a0-cb9c-461c-a0cb-e4abda95e050 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:21.519391 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cbead2-134d-4827-98c1-37d336cc6f89 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:21.544928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab7285-8278-4a60-b86a-c372acc0780d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:21.551908 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab7285-8278-4a60-b86a-c372acc0780d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:21.552408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab7285-8278-4a60-b86a-c372acc0780d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:21.555700 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aab7285-8278-4a60-b86a-c372acc0780d admin admin] PATH_INFO: `/v3/projects/24dc0fdb55324beca6e1af076c0fcd20` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:21.561211 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab7285-8278-4a60-b86a-c372acc0780d admin admin] RBAC: Authorizing `identity:delete_project(project_id=24dc0fdb55324beca6e1af076c0fcd20)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:21.572987 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aab7285-8278-4a60-b86a-c372acc0780d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:21.646268 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 865/1718] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:21 2022] POST /identity/v3/projects => generated 381 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:21.659685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fe8a0-cb9c-461c-a0cb-e4abda95e050 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:21.789704 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-7aab7285-8278-4a60-b86a-c372acc0780d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24dc0fdb55324beca6e1af076c0fcd20', 'request_id': 'req-7aab7285-8278-4a60-b86a-c372acc0780d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:21.820557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-7aab7285-8278-4a60-b86a-c372acc0780d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24dc0fdb55324beca6e1af076c0fcd20', 'request_id': 'req-7aab7285-8278-4a60-b86a-c372acc0780d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:21.848846 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-7aab7285-8278-4a60-b86a-c372acc0780d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24dc0fdb55324beca6e1af076c0fcd20', 'request_id': 'req-7aab7285-8278-4a60-b86a-c372acc0780d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:21.854182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:21.889914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:21.891226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-7aab7285-8278-4a60-b86a-c372acc0780d admin admin] The token cache is being invalidate because project 24dc0fdb55324beca6e1af076c0fcd20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:21.891510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-7aab7285-8278-4a60-b86a-c372acc0780d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:21.894449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-7aab7285-8278-4a60-b86a-c372acc0780d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:21.901957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 854/1719] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:21 2022] DELETE /identity/v3/projects/24dc0fdb55324beca6e1af076c0fcd20 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:21.902957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab7285-8278-4a60-b86a-c372acc0780d admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:21.912820 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-7aab7285-8278-4a60-b86a-c372acc0780d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:21.926928 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fe8a0-cb9c-461c-a0cb-e4abda95e050 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:22.141846 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad17734-a6eb-4c45-b607-16f01a60f918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:22.144052 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad17734-a6eb-4c45-b607-16f01a60f918 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:22.146646 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad17734-a6eb-4c45-b607-16f01a60f918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:22.147349 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad17734-a6eb-4c45-b607-16f01a60f918 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:22.150324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad17734-a6eb-4c45-b607-16f01a60f918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:22.157241 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad17734-a6eb-4c45-b607-16f01a60f918 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:22.169124 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aab7285-8278-4a60-b86a-c372acc0780d admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:22.187294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabaeb01-3e01-498c-a7e3-05cd41db389c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:22.193044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabaeb01-3e01-498c-a7e3-05cd41db389c admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:22.193554 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabaeb01-3e01-498c-a7e3-05cd41db389c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:22.194987 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabaeb01-3e01-498c-a7e3-05cd41db389c admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:22.196633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabaeb01-3e01-498c-a7e3-05cd41db389c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:22.199504 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 866/1720] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 553 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:22.203470 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad935f6-6c2b-4d0c-af10-7a7722df64f0 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:22.207194 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad935f6-6c2b-4d0c-af10-7a7722df64f0 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:22.207416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabaeb01-3e01-498c-a7e3-05cd41db389c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:22.208266 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad935f6-6c2b-4d0c-af10-7a7722df64f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:22.268360 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 855/1721] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:21 2022] POST /identity/v3/users => generated 297 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:22.269847 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabaeb01-3e01-498c-a7e3-05cd41db389c admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:22.271511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-1ad935f6-6c2b-4d0c-af10-7a7722df64f0 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:22.273345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-1ad935f6-6c2b-4d0c-af10-7a7722df64f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:22.278443 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-fabaeb01-3e01-498c-a7e3-05cd41db389c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:22.293224 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 867/1722] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:22.294369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad935f6-6c2b-4d0c-af10-7a7722df64f0 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:22.342122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad935f6-6c2b-4d0c-af10-7a7722df64f0 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:22.357858 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f4b27e-d62d-4717-b2aa-f01fa99de6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:22.358426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4b27e-d62d-4717-b2aa-f01fa99de6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:22.358748 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4b27e-d62d-4717-b2aa-f01fa99de6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:22.359122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4b27e-d62d-4717-b2aa-f01fa99de6a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:22.360824 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f4b27e-d62d-4717-b2aa-f01fa99de6a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:22.363415 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabaeb01-3e01-498c-a7e3-05cd41db389c admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:22.371479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f4b27e-d62d-4717-b2aa-f01fa99de6a1 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:22.401462 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 868/1723] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:22.409446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230cf88-379f-4158-968d-f3091c1d4ee8 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:22.411821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230cf88-379f-4158-968d-f3091c1d4ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:22.412588 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230cf88-379f-4158-968d-f3091c1d4ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:22.416121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea82bef-5ab4-42e9-92a8-90c8427f3aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:22.417335 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea82bef-5ab4-42e9-92a8-90c8427f3aba admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:22.426462 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea82bef-5ab4-42e9-92a8-90c8427f3aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:22.429419 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea82bef-5ab4-42e9-92a8-90c8427f3aba admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:22.434340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea82bef-5ab4-42e9-92a8-90c8427f3aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:22.455150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea82bef-5ab4-42e9-92a8-90c8427f3aba admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:22.467112 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-5230cf88-379f-4158-968d-f3091c1d4ee8 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:22.473280 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-5230cf88-379f-4158-968d-f3091c1d4ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:22.498277 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 869/1724] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:22.503744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230cf88-379f-4158-968d-f3091c1d4ee8 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:22.511785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 856/1725] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:22.515354 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32901f64-5008-4c47-9bd1-2364c1fc0d2a None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:22.516829 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32901f64-5008-4c47-9bd1-2364c1fc0d2a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:22.518762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32901f64-5008-4c47-9bd1-2364c1fc0d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:22.560343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230cf88-379f-4158-968d-f3091c1d4ee8 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:22.592753 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d2494-c280-4174-bf0e-2c3f5541b77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:22.593463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d2494-c280-4174-bf0e-2c3f5541b77b admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:22.593747 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d2494-c280-4174-bf0e-2c3f5541b77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:22.594136 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d2494-c280-4174-bf0e-2c3f5541b77b admin admin] PATH_INFO: `/v3/projects/02d5eeb719a64074a7ab459e2652fd28/users/d181ea0863e642668e437300cb41206e/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:22.647073 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d2494-c280-4174-bf0e-2c3f5541b77b admin admin] RBAC: Authorizing `identity:create_grant(project_id=02d5eeb719a64074a7ab459e2652fd28, user_id=d181ea0863e642668e437300cb41206e, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:22.657824 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d2494-c280-4174-bf0e-2c3f5541b77b admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:22.708178 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 870/1726] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:22 2022] PUT /identity/v3/projects/02d5eeb719a64074a7ab459e2652fd28/users/d181ea0863e642668e437300cb41206e/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:22.708393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-32901f64-5008-4c47-9bd1-2364c1fc0d2a None None] MFA Rules not processed for user `61ff29258a214817b8fc15b51fe8b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:22.708486 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d2494-c280-4174-bf0e-2c3f5541b77b admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:22.708745 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-32901f64-5008-4c47-9bd1-2364c1fc0d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:22.812148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 857/1727] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Aug 7 19:26:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:22.813330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32901f64-5008-4c47-9bd1-2364c1fc0d2a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:22.906301 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:22.972438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d2494-c280-4174-bf0e-2c3f5541b77b admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:22.973796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32901f64-5008-4c47-9bd1-2364c1fc0d2a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:22.988100 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679d0a1-3d7c-4110-930d-ee46726727aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:22.988941 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679d0a1-3d7c-4110-930d-ee46726727aa admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:22.989208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679d0a1-3d7c-4110-930d-ee46726727aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:22.989584 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679d0a1-3d7c-4110-930d-ee46726727aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:22.991300 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679d0a1-3d7c-4110-930d-ee46726727aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:26:22.991867 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679d0a1-3d7c-4110-930d-ee46726727aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:22.999388 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679d0a1-3d7c-4110-930d-ee46726727aa admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:23.021328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f80da-29ce-4256-892a-a51ba7ef0393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:23.022050 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f80da-29ce-4256-892a-a51ba7ef0393 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:23.022330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f80da-29ce-4256-892a-a51ba7ef0393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:23.022605 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f80da-29ce-4256-892a-a51ba7ef0393 admin admin] PATH_INFO: `/v3/projects/8d5f0c98cce84b379ceac995b2d09e33/users/5d06016f72274594809d95c92f7c7946/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:23.073337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 871/1728] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:26:23.080397 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679d0a1-3d7c-4110-930d-ee46726727aa admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:23.101316 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f80da-29ce-4256-892a-a51ba7ef0393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d5f0c98cce84b379ceac995b2d09e33, user_id=5d06016f72274594809d95c92f7c7946, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:23.104577 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f80da-29ce-4256-892a-a51ba7ef0393 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:23.128944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679d0a1-3d7c-4110-930d-ee46726727aa admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:23.140642 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 858/1729] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:22 2022] PUT /identity/v3/projects/8d5f0c98cce84b379ceac995b2d09e33/users/5d06016f72274594809d95c92f7c7946/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:23.141934 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f80da-29ce-4256-892a-a51ba7ef0393 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:23.264714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:23.278048 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f80da-29ce-4256-892a-a51ba7ef0393 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:23.287138 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d0619-60a9-4582-886c-d25df7ea8dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:23.287786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d0619-60a9-4582-886c-d25df7ea8dca admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:23.288454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d0619-60a9-4582-886c-d25df7ea8dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:23.288454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d0619-60a9-4582-886c-d25df7ea8dca admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:23.289030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d0619-60a9-4582-886c-d25df7ea8dca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:23.291071 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d0619-60a9-4582-886c-d25df7ea8dca admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:23.344636 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 859/1730] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:23 2022] POST /identity/v3/projects => generated 371 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:23.345600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d0619-60a9-4582-886c-d25df7ea8dca admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:23.351194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-c26d0619-60a9-4582-886c-d25df7ea8dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:23.353020 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae139d2-58df-4f73-af5c-6373a32db174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:23.354412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae139d2-58df-4f73-af5c-6373a32db174 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:23.354942 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae139d2-58df-4f73-af5c-6373a32db174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:23.356353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae139d2-58df-4f73-af5c-6373a32db174 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:23.358491 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae139d2-58df-4f73-af5c-6373a32db174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:23.361764 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae139d2-58df-4f73-af5c-6373a32db174 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:23.416104 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 872/1731] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:23.425183 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5bdba-e45e-4aa8-93b7-374662515a5c None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:23.426270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5bdba-e45e-4aa8-93b7-374662515a5c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:23.428132 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5bdba-e45e-4aa8-93b7-374662515a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:23.553259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-85e5bdba-e45e-4aa8-93b7-374662515a5c None None] MFA Rules not processed for user `5d06016f72274594809d95c92f7c7946`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:23.560568 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-85e5bdba-e45e-4aa8-93b7-374662515a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:23.643264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d0619-60a9-4582-886c-d25df7ea8dca admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:23.674209 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956bdb3-674f-48b3-b48b-457859c250de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db51971158a946c98a8ed716ecc5d72b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:23.678750 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956bdb3-674f-48b3-b48b-457859c250de service swift] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:23.679815 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956bdb3-674f-48b3-b48b-457859c250de service swift] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:23.680441 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956bdb3-674f-48b3-b48b-457859c250de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:23.738206 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:23.813650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956bdb3-674f-48b3-b48b-457859c250de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:23.813650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956bdb3-674f-48b3-b48b-457859c250de service swift] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:23.814690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 873/1732] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:23.819970 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5bdba-e45e-4aa8-93b7-374662515a5c None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:23.836516 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 860/1733] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Aug 7 19:26:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 19:26:23.839802 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956bdb3-674f-48b3-b48b-457859c250de service swift] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:24.217383 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956bdb3-674f-48b3-b48b-457859c250de service swift] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:24.250630 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abca85-6855-4d86-8b6b-a0470fa26472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:24.252257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abca85-6855-4d86-8b6b-a0470fa26472 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:24.252975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abca85-6855-4d86-8b6b-a0470fa26472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:24.253281 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abca85-6855-4d86-8b6b-a0470fa26472 admin admin] PATH_INFO: `/v3/projects/02d5eeb719a64074a7ab459e2652fd28/users/d181ea0863e642668e437300cb41206e/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:24.259761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abca85-6855-4d86-8b6b-a0470fa26472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02d5eeb719a64074a7ab459e2652fd28, user_id=d181ea0863e642668e437300cb41206e, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:24.267804 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abca85-6855-4d86-8b6b-a0470fa26472 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:24.271024 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5bdba-e45e-4aa8-93b7-374662515a5c None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:24.305784 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 861/1734] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:23 2022] PUT /identity/v3/projects/02d5eeb719a64074a7ab459e2652fd28/users/d181ea0863e642668e437300cb41206e/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:24.306443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a021e3-c2ea-4bd2-8cc5-c8bac04d6de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:24.306443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a021e3-c2ea-4bd2-8cc5-c8bac04d6de4 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:24.306443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a021e3-c2ea-4bd2-8cc5-c8bac04d6de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:24.306443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a021e3-c2ea-4bd2-8cc5-c8bac04d6de4 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:24.306791 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a021e3-c2ea-4bd2-8cc5-c8bac04d6de4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:24.307199 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abca85-6855-4d86-8b6b-a0470fa26472 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:24.315286 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a021e3-c2ea-4bd2-8cc5-c8bac04d6de4 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:24.412357 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 874/1735] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:23 2022] POST /identity/v3/users => generated 286 bytes in 597 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:24.418656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4da21a-efb3-4b9b-a461-e988fb0009a2 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:24.419382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4da21a-efb3-4b9b-a461-e988fb0009a2 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:24.420392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4da21a-efb3-4b9b-a461-e988fb0009a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:24.466979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-8d4da21a-efb3-4b9b-a461-e988fb0009a2 None None] MFA Rules not processed for user `6732487041c54918b1164b1cb72bb07e`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:24.468756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8d4da21a-efb3-4b9b-a461-e988fb0009a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:24.549405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abca85-6855-4d86-8b6b-a0470fa26472 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:24.567827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73017d8c-7d54-4a25-9b58-940b180fe565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:24.568551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73017d8c-7d54-4a25-9b58-940b180fe565 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:24.568821 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73017d8c-7d54-4a25-9b58-940b180fe565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:24.569101 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73017d8c-7d54-4a25-9b58-940b180fe565 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:24.569949 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73017d8c-7d54-4a25-9b58-940b180fe565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:24.571766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73017d8c-7d54-4a25-9b58-940b180fe565 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:24.627249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:24.641163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 862/1736] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:24 2022] POST /identity/v3/projects => generated 367 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:24.642677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73017d8c-7d54-4a25-9b58-940b180fe565 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:24.647857 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-73017d8c-7d54-4a25-9b58-940b180fe565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:24.649747 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 875/1737] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:24.651628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce443f-74f0-42b2-8bae-5a7ca7683a03 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:24.654458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce443f-74f0-42b2-8bae-5a7ca7683a03 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:24.655020 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce443f-74f0-42b2-8bae-5a7ca7683a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:24.708535 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-71ce443f-74f0-42b2-8bae-5a7ca7683a03 None None] MFA Rules not processed for user `d181ea0863e642668e437300cb41206e`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:24.710209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-71ce443f-74f0-42b2-8bae-5a7ca7683a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:24.836552 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 876/1738] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:24.838124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce443f-74f0-42b2-8bae-5a7ca7683a03 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:24.858123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:24.928641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73017d8c-7d54-4a25-9b58-940b180fe565 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:24.937150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d423232-6c25-49a4-a340-374adaba1352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:24.937962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d423232-6c25-49a4-a340-374adaba1352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:24.938458 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d423232-6c25-49a4-a340-374adaba1352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:24.949608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d423232-6c25-49a4-a340-374adaba1352 admin admin] PATH_INFO: `/v3/users/d536120c684e405fb96766b87764ec9c` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:24.951788 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d423232-6c25-49a4-a340-374adaba1352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d536120c684e405fb96766b87764ec9c)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:24.963484 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d423232-6c25-49a4-a340-374adaba1352 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:25.180643 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1d423232-6c25-49a4-a340-374adaba1352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd536120c684e405fb96766b87764ec9c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:25.255985 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce443f-74f0-42b2-8bae-5a7ca7683a03 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:25.257938 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1d423232-6c25-49a4-a340-374adaba1352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd536120c684e405fb96766b87764ec9c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:25.285945 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee29a5-ab31-45e1-876d-3def48ddad54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:25.286590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee29a5-ab31-45e1-876d-3def48ddad54 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:25.286869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee29a5-ab31-45e1-876d-3def48ddad54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:25.287143 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee29a5-ab31-45e1-876d-3def48ddad54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:25.287801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee29a5-ab31-45e1-876d-3def48ddad54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:25.294352 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1d423232-6c25-49a4-a340-374adaba1352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd536120c684e405fb96766b87764ec9c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:25.295150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1d423232-6c25-49a4-a340-374adaba1352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd536120c684e405fb96766b87764ec9c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:25.295780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1d423232-6c25-49a4-a340-374adaba1352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd536120c684e405fb96766b87764ec9c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:25.305246 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee29a5-ab31-45e1-876d-3def48ddad54 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:25.348673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 863/1739] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:24 2022] DELETE /identity/v3/users/d536120c684e405fb96766b87764ec9c => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:25.355796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d423232-6c25-49a4-a340-374adaba1352 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:25.356817 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 877/1740] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 519 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:25.357722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee29a5-ab31-45e1-876d-3def48ddad54 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:25.358871 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1d423232-6c25-49a4-a340-374adaba1352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:25.373179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-79ee29a5-ab31-45e1-876d-3def48ddad54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:25.536753 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee29a5-ab31-45e1-876d-3def48ddad54 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:25.593670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:25.632868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90499f62-5b08-4726-b3b4-54efa74641e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:25.633585 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90499f62-5b08-4726-b3b4-54efa74641e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:25.634070 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90499f62-5b08-4726-b3b4-54efa74641e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:25.634357 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90499f62-5b08-4726-b3b4-54efa74641e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:25.636618 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-90499f62-5b08-4726-b3b4-54efa74641e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:25.677085 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d423232-6c25-49a4-a340-374adaba1352 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:25.692374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14355f-2432-4d03-b39a-9e24f896e08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:25.693020 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14355f-2432-4d03-b39a-9e24f896e08e admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:25.693298 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14355f-2432-4d03-b39a-9e24f896e08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:25.693588 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14355f-2432-4d03-b39a-9e24f896e08e admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:25.694215 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14355f-2432-4d03-b39a-9e24f896e08e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:25.701659 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14355f-2432-4d03-b39a-9e24f896e08e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:25.754049 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 864/1741] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:25 2022] POST /identity/v3/users => generated 290 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:25.756899 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c395e-2923-448a-a34f-1ef3c229adcd None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:25.759582 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c395e-2923-448a-a34f-1ef3c229adcd None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:25.761576 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c395e-2923-448a-a34f-1ef3c229adcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:25.792298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90499f62-5b08-4726-b3b4-54efa74641e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:25.799135 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90499f62-5b08-4726-b3b4-54efa74641e3 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:25.818023 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-4a0c395e-2923-448a-a34f-1ef3c229adcd None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:25.820622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4a0c395e-2923-448a-a34f-1ef3c229adcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:25.845738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 878/1742] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:25.847296 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90499f62-5b08-4726-b3b4-54efa74641e3 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:25.849519 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-90499f62-5b08-4726-b3b4-54efa74641e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:25.851185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 865/1743] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:25.855848 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c395e-2923-448a-a34f-1ef3c229adcd None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:25.888685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90499f62-5b08-4726-b3b4-54efa74641e3 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:25.894165 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c395e-2923-448a-a34f-1ef3c229adcd None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:25.895124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149757c9-5be5-481f-b5aa-be74c8038626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:25.895835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149757c9-5be5-481f-b5aa-be74c8038626 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:25.896296 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149757c9-5be5-481f-b5aa-be74c8038626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:25.896660 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149757c9-5be5-481f-b5aa-be74c8038626 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:25.897508 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149757c9-5be5-481f-b5aa-be74c8038626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:25.899523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149757c9-5be5-481f-b5aa-be74c8038626 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:25.927487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31920eb7-50b0-455b-b400-1b9ab11b3629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:25.927487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31920eb7-50b0-455b-b400-1b9ab11b3629 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:25.927487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31920eb7-50b0-455b-b400-1b9ab11b3629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:25.927487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31920eb7-50b0-455b-b400-1b9ab11b3629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:25.930105 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31920eb7-50b0-455b-b400-1b9ab11b3629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:25.931919 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31920eb7-50b0-455b-b400-1b9ab11b3629 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:25.943722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 879/1744] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:25.943722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149757c9-5be5-481f-b5aa-be74c8038626 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:25.947881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-149757c9-5be5-481f-b5aa-be74c8038626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:25.988660 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 866/1745] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:25.989739 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31920eb7-50b0-455b-b400-1b9ab11b3629 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:26.029752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31920eb7-50b0-455b-b400-1b9ab11b3629 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:26.043615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cee7e8-6c5a-46df-83d8-17ebb84a5528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:26.044362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cee7e8-6c5a-46df-83d8-17ebb84a5528 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:26.044873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cee7e8-6c5a-46df-83d8-17ebb84a5528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:26.044873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cee7e8-6c5a-46df-83d8-17ebb84a5528 admin admin] PATH_INFO: `/v3/projects/d6007723a068405d9032c43177fe5097/users/a0b2a4d397f14de2b8164761d03a1258/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:26.078081 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cee7e8-6c5a-46df-83d8-17ebb84a5528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6007723a068405d9032c43177fe5097, user_id=a0b2a4d397f14de2b8164761d03a1258, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:26.082677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cee7e8-6c5a-46df-83d8-17ebb84a5528 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:26.146717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 867/1746] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:25 2022] PUT /identity/v3/projects/d6007723a068405d9032c43177fe5097/users/a0b2a4d397f14de2b8164761d03a1258/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:26.147940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cee7e8-6c5a-46df-83d8-17ebb84a5528 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:26.230326 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:26.321703 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149757c9-5be5-481f-b5aa-be74c8038626 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:26.405490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cee7e8-6c5a-46df-83d8-17ebb84a5528 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:26.433794 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e935a-1435-4b58-a73c-65a324dd0dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:26.435573 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e935a-1435-4b58-a73c-65a324dd0dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:26.436079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e935a-1435-4b58-a73c-65a324dd0dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:26.436519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e935a-1435-4b58-a73c-65a324dd0dcf admin admin] PATH_INFO: `/v3/projects/c992244de13c4f0bb6244e4f9ea53001/users/87602e0463c94d50821ca952871aaaa2/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:26.511847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684e310-8f50-4582-9a0e-672e043dc81f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:26.512677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684e310-8f50-4582-9a0e-672e043dc81f service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:26.513065 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684e310-8f50-4582-9a0e-672e043dc81f service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:26.513451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684e310-8f50-4582-9a0e-672e043dc81f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:26.517926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e935a-1435-4b58-a73c-65a324dd0dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c992244de13c4f0bb6244e4f9ea53001, user_id=87602e0463c94d50821ca952871aaaa2, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:26.526708 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e935a-1435-4b58-a73c-65a324dd0dcf admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:26.542731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684e310-8f50-4582-9a0e-672e043dc81f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:26.544579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d684e310-8f50-4582-9a0e-672e043dc81f service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:26.573968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 880/1747] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:26.575040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684e310-8f50-4582-9a0e-672e043dc81f service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:26.631536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 868/1748] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:26 2022] PUT /identity/v3/projects/c992244de13c4f0bb6244e4f9ea53001/users/87602e0463c94d50821ca952871aaaa2/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:26.646012 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581afd3-f683-4a7c-835f-bbc6f1f143b9 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:26.647258 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581afd3-f683-4a7c-835f-bbc6f1f143b9 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:26.648700 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581afd3-f683-4a7c-835f-bbc6f1f143b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:26.707769 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:26.763242 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d684e310-8f50-4582-9a0e-672e043dc81f service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:26.830495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-0581afd3-f683-4a7c-835f-bbc6f1f143b9 None None] MFA Rules not processed for user `a0b2a4d397f14de2b8164761d03a1258`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:26.831777 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-0581afd3-f683-4a7c-835f-bbc6f1f143b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:26.986467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d516d0-897c-4404-b071-8d6d0bc9bf3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:26.987456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d516d0-897c-4404-b071-8d6d0bc9bf3f service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:26.987840 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d516d0-897c-4404-b071-8d6d0bc9bf3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:26.988244 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d516d0-897c-4404-b071-8d6d0bc9bf3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:26.999054 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-89d516d0-897c-4404-b071-8d6d0bc9bf3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:27.108756 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 869/1749] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:27.108756 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581afd3-f683-4a7c-835f-bbc6f1f143b9 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:27.186573 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d516d0-897c-4404-b071-8d6d0bc9bf3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:27.195806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d516d0-897c-4404-b071-8d6d0bc9bf3f service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:27.233085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 881/1750] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:27.300722 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581afd3-f683-4a7c-835f-bbc6f1f143b9 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:27.310636 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c785f-972e-455b-98d0-65e11e358f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:27.311660 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c785f-972e-455b-98d0-65e11e358f01 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:27.312423 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c785f-972e-455b-98d0-65e11e358f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:27.312855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c785f-972e-455b-98d0-65e11e358f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:27.313709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c785f-972e-455b-98d0-65e11e358f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:27.316088 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c785f-972e-455b-98d0-65e11e358f01 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:27.340591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 870/1751] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:27.343440 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d516d0-897c-4404-b071-8d6d0bc9bf3f service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:27.388161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d516d0-897c-4404-b071-8d6d0bc9bf3f service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:27.423981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72bf60-f95d-4f09-8bd5-00562e591727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:27.425058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72bf60-f95d-4f09-8bd5-00562e591727 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:27.425608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72bf60-f95d-4f09-8bd5-00562e591727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:27.426096 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72bf60-f95d-4f09-8bd5-00562e591727 admin admin] PATH_INFO: `/v3/projects/c992244de13c4f0bb6244e4f9ea53001/users/87602e0463c94d50821ca952871aaaa2/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:27.434681 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72bf60-f95d-4f09-8bd5-00562e591727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c992244de13c4f0bb6244e4f9ea53001, user_id=87602e0463c94d50821ca952871aaaa2, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:27.445167 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72bf60-f95d-4f09-8bd5-00562e591727 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:27.516075 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 882/1752] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:27 2022] PUT /identity/v3/projects/c992244de13c4f0bb6244e4f9ea53001/users/87602e0463c94d50821ca952871aaaa2/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:27.524550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70044305-3cfd-46ba-b39a-6d69b548e63b None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:27.526429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70044305-3cfd-46ba-b39a-6d69b548e63b None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:27.527599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70044305-3cfd-46ba-b39a-6d69b548e63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:27.539380 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02af8e-ef84-4c47-a66e-2224ecd61a62 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:27.543489 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02af8e-ef84-4c47-a66e-2224ecd61a62 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:27.546882 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02af8e-ef84-4c47-a66e-2224ecd61a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:27.607415 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-70044305-3cfd-46ba-b39a-6d69b548e63b None None] MFA Rules not processed for user `08229e9400b14e8a9cc8e8c32dc48285`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:27.621003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-70044305-3cfd-46ba-b39a-6d69b548e63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:27.695399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-0b02af8e-ef84-4c47-a66e-2224ecd61a62 None None] MFA Rules not processed for user `87602e0463c94d50821ca952871aaaa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:27.699658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0b02af8e-ef84-4c47-a66e-2224ecd61a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:27.729374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:27.921380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 871/1753] 10.222.0.18 () {58 vars in 1015 bytes} [Sun Aug 7 19:26:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:26:28.029594 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70044305-3cfd-46ba-b39a-6d69b548e63b None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:28.032105 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-70044305-3cfd-46ba-b39a-6d69b548e63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:28.066310 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70044305-3cfd-46ba-b39a-6d69b548e63b None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:28.088025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f2979-fcb6-487e-be51-08952fb1a499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:28.088886 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f2979-fcb6-487e-be51-08952fb1a499 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:28.089479 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f2979-fcb6-487e-be51-08952fb1a499 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:28.090016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0f2979-fcb6-487e-be51-08952fb1a499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:28.105684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f2979-fcb6-487e-be51-08952fb1a499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:28.113631 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0f2979-fcb6-487e-be51-08952fb1a499 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:28.128395 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 872/1754] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:28.128660 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 883/1755] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 593 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:28.142299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02af8e-ef84-4c47-a66e-2224ecd61a62 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:28.156861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f2979-fcb6-487e-be51-08952fb1a499 service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:28.176886 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0f2979-fcb6-487e-be51-08952fb1a499 service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:28.192305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c929f3b-a44c-4ec5-a55e-84dc7d41161a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:28.192804 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c929f3b-a44c-4ec5-a55e-84dc7d41161a service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:28.196951 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c929f3b-a44c-4ec5-a55e-84dc7d41161a service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:28.197600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c929f3b-a44c-4ec5-a55e-84dc7d41161a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:28.218612 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:28.232535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c929f3b-a44c-4ec5-a55e-84dc7d41161a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:28.234729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c929f3b-a44c-4ec5-a55e-84dc7d41161a service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:28.259211 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 873/1756] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:26:28.398543 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02af8e-ef84-4c47-a66e-2224ecd61a62 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:28.411140 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72dfb6a-b0ec-4a26-a24d-279f4538ae38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:28.418318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dfb6a-b0ec-4a26-a24d-279f4538ae38 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:28.419154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dfb6a-b0ec-4a26-a24d-279f4538ae38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:28.422346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dfb6a-b0ec-4a26-a24d-279f4538ae38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:28.458474 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72dfb6a-b0ec-4a26-a24d-279f4538ae38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:28.460437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72dfb6a-b0ec-4a26-a24d-279f4538ae38 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:28.485955 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c929f3b-a44c-4ec5-a55e-84dc7d41161a service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:28.489327 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 884/1757] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:28.497402 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-3c929f3b-a44c-4ec5-a55e-84dc7d41161a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:28.752893 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c929f3b-a44c-4ec5-a55e-84dc7d41161a service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:28.763318 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb6c5e-284c-432c-bbe0-286f904bda52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:28.765149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb6c5e-284c-432c-bbe0-286f904bda52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:28.766273 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb6c5e-284c-432c-bbe0-286f904bda52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:28.767418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb6c5e-284c-432c-bbe0-286f904bda52 admin admin] PATH_INFO: `/v3/users/ee32aee54d884eccaf17c81853f214f3` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:28.795485 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb6c5e-284c-432c-bbe0-286f904bda52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee32aee54d884eccaf17c81853f214f3)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:28.816280 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb6c5e-284c-432c-bbe0-286f904bda52 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:28.936618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-fbfb6c5e-284c-432c-bbe0-286f904bda52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee32aee54d884eccaf17c81853f214f3'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:28.938086 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72dfb6a-b0ec-4a26-a24d-279f4538ae38 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:28.941779 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f72dfb6a-b0ec-4a26-a24d-279f4538ae38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:28.959896 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-fbfb6c5e-284c-432c-bbe0-286f904bda52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee32aee54d884eccaf17c81853f214f3'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:29.000201 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-fbfb6c5e-284c-432c-bbe0-286f904bda52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee32aee54d884eccaf17c81853f214f3'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:29.001365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-fbfb6c5e-284c-432c-bbe0-286f904bda52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee32aee54d884eccaf17c81853f214f3'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:29.004726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-fbfb6c5e-284c-432c-bbe0-286f904bda52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee32aee54d884eccaf17c81853f214f3'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:29.087461 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 874/1758] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:28 2022] DELETE /identity/v3/users/ee32aee54d884eccaf17c81853f214f3 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:29.091580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad1138-f64a-4f77-b082-df07ded1b736 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:29.091580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad1138-f64a-4f77-b082-df07ded1b736 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:29.091877 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad1138-f64a-4f77-b082-df07ded1b736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:29.192816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-f5ad1138-f64a-4f77-b082-df07ded1b736 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:29.196912 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f5ad1138-f64a-4f77-b082-df07ded1b736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:29.211957 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72dfb6a-b0ec-4a26-a24d-279f4538ae38 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:29.310359 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:29.500434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff1d29-9762-4fa1-86f2-b645a29f069a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:29.501386 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff1d29-9762-4fa1-86f2-b645a29f069a service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:29.501885 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff1d29-9762-4fa1-86f2-b645a29f069a service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:29.502313 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff1d29-9762-4fa1-86f2-b645a29f069a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:29.507850 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 875/1759] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:29.517799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-23ff1d29-9762-4fa1-86f2-b645a29f069a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:29.727399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:29.738385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff1d29-9762-4fa1-86f2-b645a29f069a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:29.747195 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff1d29-9762-4fa1-86f2-b645a29f069a service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:29.788144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 885/1760] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 852 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:29.907083 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad1138-f64a-4f77-b082-df07ded1b736 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:29.914842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f5ad1138-f64a-4f77-b082-df07ded1b736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:30.242039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad1138-f64a-4f77-b082-df07ded1b736 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:30.243910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff1d29-9762-4fa1-86f2-b645a29f069a service nova] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:30.246744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-23ff1d29-9762-4fa1-86f2-b645a29f069a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:30.288405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa162daa-5230-4d2e-9a69-76d798ab4663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:30.292106 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa162daa-5230-4d2e-9a69-76d798ab4663 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:30.292469 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa162daa-5230-4d2e-9a69-76d798ab4663 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:30.295205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa162daa-5230-4d2e-9a69-76d798ab4663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:30.344662 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff1d29-9762-4fa1-86f2-b645a29f069a service nova] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:30.379408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa162daa-5230-4d2e-9a69-76d798ab4663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:30.385059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f8a36-a357-41a2-8b39-74a98a9d5a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:30.388976 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f8a36-a357-41a2-8b39-74a98a9d5a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:30.390908 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f8a36-a357-41a2-8b39-74a98a9d5a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:30.393249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f8a36-a357-41a2-8b39-74a98a9d5a47 admin admin] PATH_INFO: `/v3/projects/7adc672f1cfd4f31a962a47097b7d073` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:30.393798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa162daa-5230-4d2e-9a69-76d798ab4663 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:30.404108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f8a36-a357-41a2-8b39-74a98a9d5a47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7adc672f1cfd4f31a962a47097b7d073)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:30.421119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f8a36-a357-41a2-8b39-74a98a9d5a47 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:30.477925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 876/1761] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:26:30.775806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-940f8a36-a357-41a2-8b39-74a98a9d5a47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7adc672f1cfd4f31a962a47097b7d073', 'request_id': 'req-940f8a36-a357-41a2-8b39-74a98a9d5a47'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:30.808607 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-940f8a36-a357-41a2-8b39-74a98a9d5a47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7adc672f1cfd4f31a962a47097b7d073', 'request_id': 'req-940f8a36-a357-41a2-8b39-74a98a9d5a47'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:30.853619 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-940f8a36-a357-41a2-8b39-74a98a9d5a47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7adc672f1cfd4f31a962a47097b7d073', 'request_id': 'req-940f8a36-a357-41a2-8b39-74a98a9d5a47'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:30.930530 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-940f8a36-a357-41a2-8b39-74a98a9d5a47 admin admin] The token cache is being invalidate because project 7adc672f1cfd4f31a962a47097b7d073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:30.932671 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-940f8a36-a357-41a2-8b39-74a98a9d5a47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:30.936360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-940f8a36-a357-41a2-8b39-74a98a9d5a47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:30.943421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 886/1762] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:30 2022] DELETE /identity/v3/projects/7adc672f1cfd4f31a962a47097b7d073 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:30.949726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e260fa3-ea5d-4665-8cb9-08a64eb763c4 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:30.950773 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e260fa3-ea5d-4665-8cb9-08a64eb763c4 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:30.951262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e260fa3-ea5d-4665-8cb9-08a64eb763c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:31.052988 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-2e260fa3-ea5d-4665-8cb9-08a64eb763c4 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:31.060421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-2e260fa3-ea5d-4665-8cb9-08a64eb763c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:31.299863 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 877/1763] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:31.309410 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb1d19-0faf-4198-b53e-1193122ba34a None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:31.310657 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb1d19-0faf-4198-b53e-1193122ba34a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:31.311220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb1d19-0faf-4198-b53e-1193122ba34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:31.362873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-38eb1d19-0faf-4198-b53e-1193122ba34a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:31.366099 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-38eb1d19-0faf-4198-b53e-1193122ba34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:31.393372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 887/1764] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:26:31.397635 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e260fa3-ea5d-4665-8cb9-08a64eb763c4 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:31.413758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e260fa3-ea5d-4665-8cb9-08a64eb763c4 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:31.429135 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b2ea3-e2db-4381-82e8-c7f491f19e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:31.429958 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b2ea3-e2db-4381-82e8-c7f491f19e11 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:31.430248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b2ea3-e2db-4381-82e8-c7f491f19e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:31.432885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b2ea3-e2db-4381-82e8-c7f491f19e11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:31.433698 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b2ea3-e2db-4381-82e8-c7f491f19e11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:26:31.434078 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b2ea3-e2db-4381-82e8-c7f491f19e11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:31.440641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b2ea3-e2db-4381-82e8-c7f491f19e11 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:31.472161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 878/1765] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:26:31.477629 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb1d19-0faf-4198-b53e-1193122ba34a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:31.489657 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb1d19-0faf-4198-b53e-1193122ba34a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:31.509065 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd77b48-85fb-4217-88f1-d0651b097837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:31.509818 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd77b48-85fb-4217-88f1-d0651b097837 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:31.510521 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd77b48-85fb-4217-88f1-d0651b097837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:31.510903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd77b48-85fb-4217-88f1-d0651b097837 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:31.511740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd77b48-85fb-4217-88f1-d0651b097837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:31.518675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd77b48-85fb-4217-88f1-d0651b097837 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:31.612244 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 888/1766] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:31 2022] POST /identity/v3/projects => generated 369 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:31.617131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b2ea3-e2db-4381-82e8-c7f491f19e11 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:31.642568 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b2ea3-e2db-4381-82e8-c7f491f19e11 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:31.748729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242d98e-687a-47bf-8a9b-9489d00e9464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:31.749422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242d98e-687a-47bf-8a9b-9489d00e9464 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:31.749707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242d98e-687a-47bf-8a9b-9489d00e9464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:31.749984 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242d98e-687a-47bf-8a9b-9489d00e9464 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:31.751333 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242d98e-687a-47bf-8a9b-9489d00e9464 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:31.754043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242d98e-687a-47bf-8a9b-9489d00e9464 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:31.769510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:31.826459 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 879/1767] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:31 2022] POST /identity/v3/users => generated 285 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:31.827945 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd77b48-85fb-4217-88f1-d0651b097837 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:31.856476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd77b48-85fb-4217-88f1-d0651b097837 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:31.890555 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522a073-ba74-4be9-8724-f7ce35e55a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:31.891273 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522a073-ba74-4be9-8724-f7ce35e55a85 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:31.891724 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522a073-ba74-4be9-8724-f7ce35e55a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:31.892065 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522a073-ba74-4be9-8724-f7ce35e55a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:31.892805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522a073-ba74-4be9-8724-f7ce35e55a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:31.900540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522a073-ba74-4be9-8724-f7ce35e55a85 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:31.945364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242d98e-687a-47bf-8a9b-9489d00e9464 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:31.945636 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 889/1768] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:26:31.966375 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242d98e-687a-47bf-8a9b-9489d00e9464 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:31.978373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92f5aa-735e-4fdc-9a94-aaf16db514ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:31.979743 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92f5aa-735e-4fdc-9a94-aaf16db514ce admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:31.980241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92f5aa-735e-4fdc-9a94-aaf16db514ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:31.981418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92f5aa-735e-4fdc-9a94-aaf16db514ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:31.982994 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92f5aa-735e-4fdc-9a94-aaf16db514ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:31.998499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92f5aa-735e-4fdc-9a94-aaf16db514ce admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:32.015817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 880/1769] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:32.020321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522a073-ba74-4be9-8724-f7ce35e55a85 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:32.034079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522a073-ba74-4be9-8724-f7ce35e55a85 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:32.052691 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef33ee-ffbb-437d-8cbb-08bd41d31241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:32.053496 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef33ee-ffbb-437d-8cbb-08bd41d31241 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:32.053783 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef33ee-ffbb-437d-8cbb-08bd41d31241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:32.054067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eef33ee-ffbb-437d-8cbb-08bd41d31241 admin admin] PATH_INFO: `/v3/projects/48e656da916248ef9adb2cbd9b2e6770/users/c343ea204aae49fd96bce06bebd26ad7/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:32.107952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef33ee-ffbb-437d-8cbb-08bd41d31241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48e656da916248ef9adb2cbd9b2e6770, user_id=c343ea204aae49fd96bce06bebd26ad7, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:32.110912 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eef33ee-ffbb-437d-8cbb-08bd41d31241 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:32.174981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 890/1770] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:32 2022] PUT /identity/v3/projects/48e656da916248ef9adb2cbd9b2e6770/users/c343ea204aae49fd96bce06bebd26ad7/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:32.189798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d42359-fff5-43fc-a1f1-713baaa780a5 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:32.190470 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d42359-fff5-43fc-a1f1-713baaa780a5 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:32.190975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d42359-fff5-43fc-a1f1-713baaa780a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:32.345048 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-d9d42359-fff5-43fc-a1f1-713baaa780a5 None None] MFA Rules not processed for user `c343ea204aae49fd96bce06bebd26ad7`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:32.346944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d9d42359-fff5-43fc-a1f1-713baaa780a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:32.653904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 881/1771] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:32.662834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef33ee-ffbb-437d-8cbb-08bd41d31241 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:32.889601 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eef33ee-ffbb-437d-8cbb-08bd41d31241 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:32.904305 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98471067-2f8f-4798-9f35-d53a8c621916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:32.905147 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98471067-2f8f-4798-9f35-d53a8c621916 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:32.905733 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98471067-2f8f-4798-9f35-d53a8c621916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:32.906422 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98471067-2f8f-4798-9f35-d53a8c621916 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:32.907789 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98471067-2f8f-4798-9f35-d53a8c621916 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:32.912449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98471067-2f8f-4798-9f35-d53a8c621916 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:32.953803 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:32.964432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d42359-fff5-43fc-a1f1-713baaa780a5 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:32.964633 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 891/1772] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:32 2022] POST /identity/v3/projects => generated 369 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:32.999091 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d42359-fff5-43fc-a1f1-713baaa780a5 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:33.017974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98471067-2f8f-4798-9f35-d53a8c621916 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:33.020228 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-98471067-2f8f-4798-9f35-d53a8c621916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:33.180522 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665c9e8-9e1b-48f3-992d-9684945d6d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:33.182095 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665c9e8-9e1b-48f3-992d-9684945d6d27 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:33.182647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665c9e8-9e1b-48f3-992d-9684945d6d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:33.183713 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5665c9e8-9e1b-48f3-992d-9684945d6d27 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:33.185167 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665c9e8-9e1b-48f3-992d-9684945d6d27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:33.188976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5665c9e8-9e1b-48f3-992d-9684945d6d27 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:33.221811 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 882/1773] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:32 2022] POST /identity/v3/users => generated 291 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:33.225984 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665c9e8-9e1b-48f3-992d-9684945d6d27 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:33.229498 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98471067-2f8f-4798-9f35-d53a8c621916 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:33.248971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6a34c-2eac-4166-9530-3c81c2e17ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:33.249877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6a34c-2eac-4166-9530-3c81c2e17ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:33.250345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6a34c-2eac-4166-9530-3c81c2e17ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:33.250759 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6a34c-2eac-4166-9530-3c81c2e17ebe admin admin] PATH_INFO: `/v3/users/5d06016f72274594809d95c92f7c7946` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:33.252398 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6a34c-2eac-4166-9530-3c81c2e17ebe admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d06016f72274594809d95c92f7c7946)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:33.256632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6a34c-2eac-4166-9530-3c81c2e17ebe admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:33.263734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5665c9e8-9e1b-48f3-992d-9684945d6d27 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:33.277481 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff11b8-6ebc-4f1d-b97a-d8718e4d7316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:33.278280 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff11b8-6ebc-4f1d-b97a-d8718e4d7316 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:33.278565 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff11b8-6ebc-4f1d-b97a-d8718e4d7316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:33.278900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff11b8-6ebc-4f1d-b97a-d8718e4d7316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:33.279576 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff11b8-6ebc-4f1d-b97a-d8718e4d7316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:33.281568 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff11b8-6ebc-4f1d-b97a-d8718e4d7316 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:33.324443 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 883/1774] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:33.330283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff11b8-6ebc-4f1d-b97a-d8718e4d7316 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:33.357189 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff11b8-6ebc-4f1d-b97a-d8718e4d7316 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:33.387960 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cfd6a34c-2eac-4166-9530-3c81c2e17ebe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d06016f72274594809d95c92f7c7946'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:33.390929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4782a-538a-48ee-bf31-cbdb48859567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:33.391775 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4782a-538a-48ee-bf31-cbdb48859567 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:33.392185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4782a-538a-48ee-bf31-cbdb48859567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:33.392472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4782a-538a-48ee-bf31-cbdb48859567 admin admin] PATH_INFO: `/v3/projects/d9cd15b93e4b473eac1cfd78c3c71adb/users/4ea42599832f4f34bed14620e5a5aaf5/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:33.403620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cfd6a34c-2eac-4166-9530-3c81c2e17ebe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d06016f72274594809d95c92f7c7946'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:33.487083 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4782a-538a-48ee-bf31-cbdb48859567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9cd15b93e4b473eac1cfd78c3c71adb, user_id=4ea42599832f4f34bed14620e5a5aaf5, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:33.491498 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4782a-538a-48ee-bf31-cbdb48859567 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:33.513660 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cfd6a34c-2eac-4166-9530-3c81c2e17ebe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d06016f72274594809d95c92f7c7946'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:33.520356 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cfd6a34c-2eac-4166-9530-3c81c2e17ebe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d06016f72274594809d95c92f7c7946'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:33.561944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cfd6a34c-2eac-4166-9530-3c81c2e17ebe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d06016f72274594809d95c92f7c7946'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:33.580389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 884/1775] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:33 2022] PUT /identity/v3/projects/d9cd15b93e4b473eac1cfd78c3c71adb/users/4ea42599832f4f34bed14620e5a5aaf5/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:26:33.582115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4782a-538a-48ee-bf31-cbdb48859567 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:33.587614 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 892/1776] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:33 2022] DELETE /identity/v3/users/5d06016f72274594809d95c92f7c7946 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:33.588753 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-32b4782a-538a-48ee-bf31-cbdb48859567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:33.593007 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b28d4-c742-4bc0-96b6-498cda86ae5c None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:33.593299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b28d4-c742-4bc0-96b6-498cda86ae5c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:33.593578 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b28d4-c742-4bc0-96b6-498cda86ae5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:33.694187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-433b28d4-c742-4bc0-96b6-498cda86ae5c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:33.698659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-433b28d4-c742-4bc0-96b6-498cda86ae5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:33.737886 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:33.834301 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 893/1777] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:33.901740 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4782a-538a-48ee-bf31-cbdb48859567 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:33.910795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b28d4-c742-4bc0-96b6-498cda86ae5c None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:33.914023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-433b28d4-c742-4bc0-96b6-498cda86ae5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:33.941889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18a872-4965-4829-a04c-da32e6e08ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:33.943018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18a872-4965-4829-a04c-da32e6e08ade admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:33.948413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18a872-4965-4829-a04c-da32e6e08ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:33.951572 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18a872-4965-4829-a04c-da32e6e08ade admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:33.953022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18a872-4965-4829-a04c-da32e6e08ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:33.964726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18a872-4965-4829-a04c-da32e6e08ade admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:34.022445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 885/1778] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:34.028027 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18a872-4965-4829-a04c-da32e6e08ade admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:34.029596 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:34.109818 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18a872-4965-4829-a04c-da32e6e08ade admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:34.128744 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5b7df-e9fe-4a28-a38f-37ea7438f1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:34.131791 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5b7df-e9fe-4a28-a38f-37ea7438f1e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:34.135518 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5b7df-e9fe-4a28-a38f-37ea7438f1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:34.136052 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5b7df-e9fe-4a28-a38f-37ea7438f1e0 admin admin] PATH_INFO: `/v3/projects/d9cd15b93e4b473eac1cfd78c3c71adb/users/4ea42599832f4f34bed14620e5a5aaf5/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:34.139818 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5b7df-e9fe-4a28-a38f-37ea7438f1e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9cd15b93e4b473eac1cfd78c3c71adb, user_id=4ea42599832f4f34bed14620e5a5aaf5, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:34.144982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5b7df-e9fe-4a28-a38f-37ea7438f1e0 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:34.180830 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 886/1779] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:34 2022] PUT /identity/v3/projects/d9cd15b93e4b473eac1cfd78c3c71adb/users/4ea42599832f4f34bed14620e5a5aaf5/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:34.192422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd15e7-f99d-41bf-a996-ffe58ea69c1a None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:34.192707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd15e7-f99d-41bf-a996-ffe58ea69c1a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:34.193163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd15e7-f99d-41bf-a996-ffe58ea69c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:34.198564 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b28d4-c742-4bc0-96b6-498cda86ae5c None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:34.250753 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-e8bd15e7-f99d-41bf-a996-ffe58ea69c1a None None] MFA Rules not processed for user `4ea42599832f4f34bed14620e5a5aaf5`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:34.252485 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e8bd15e7-f99d-41bf-a996-ffe58ea69c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:34.369738 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 887/1780] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:26:34.390410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd15e7-f99d-41bf-a996-ffe58ea69c1a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:34.393510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e8bd15e7-f99d-41bf-a996-ffe58ea69c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:34.405784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303da659-3b18-419a-8a51-72c89dcb4cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:34.406706 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303da659-3b18-419a-8a51-72c89dcb4cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:34.407159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303da659-3b18-419a-8a51-72c89dcb4cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:34.407569 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303da659-3b18-419a-8a51-72c89dcb4cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:34.517742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd15e7-f99d-41bf-a996-ffe58ea69c1a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:34.536903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88e2d5-3ec8-401f-a175-c0f28ee68f16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:34.536903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88e2d5-3ec8-401f-a175-c0f28ee68f16 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:34.536903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88e2d5-3ec8-401f-a175-c0f28ee68f16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:34.536903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88e2d5-3ec8-401f-a175-c0f28ee68f16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:34.560805 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88e2d5-3ec8-401f-a175-c0f28ee68f16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:34.566066 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88e2d5-3ec8-401f-a175-c0f28ee68f16 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:34.616107 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 888/1781] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:34.633261 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303da659-3b18-419a-8a51-72c89dcb4cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:34.634804 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303da659-3b18-419a-8a51-72c89dcb4cd1 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:34.690388 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 894/1782] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 782 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:34.822494 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88e2d5-3ec8-401f-a175-c0f28ee68f16 service cinder] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:34.838288 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88e2d5-3ec8-401f-a175-c0f28ee68f16 service cinder] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:34.853702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ff0b6-97cd-4b6e-a91d-1696ef2814be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:34.854445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ff0b6-97cd-4b6e-a91d-1696ef2814be service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:34.854697 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ff0b6-97cd-4b6e-a91d-1696ef2814be service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:34.854949 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ff0b6-97cd-4b6e-a91d-1696ef2814be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:34.857061 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b19ff0b6-97cd-4b6e-a91d-1696ef2814be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:34.902823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303da659-3b18-419a-8a51-72c89dcb4cd1 service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:34.904607 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-303da659-3b18-419a-8a51-72c89dcb4cd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:34.924531 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303da659-3b18-419a-8a51-72c89dcb4cd1 service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:34.940962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:34.941640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:34.941942 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:34.942204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35 admin admin] PATH_INFO: `/v3/projects/d6d71958f4f042618dce65173e4a7180` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:34.953006 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6d71958f4f042618dce65173e4a7180)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:34.955220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:34.958066 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:35.071207 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6d71958f4f042618dce65173e4a7180', 'request_id': 'req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:35.080933 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ff0b6-97cd-4b6e-a91d-1696ef2814be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:35.085463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6d71958f4f042618dce65173e4a7180', 'request_id': 'req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:35.099909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ff0b6-97cd-4b6e-a91d-1696ef2814be service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:35.100484 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6d71958f4f042618dce65173e4a7180', 'request_id': 'req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:35.125044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35 admin admin] The token cache is being invalidate because project d6d71958f4f042618dce65173e4a7180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:35.125476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:35.132494 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:35.134668 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 895/1783] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:34 2022] DELETE /identity/v3/projects/d6d71958f4f042618dce65173e4a7180 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:26:35.138632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:35.140219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:35.147574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 889/1784] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:35.187932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:35.289738 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e044f-48f3-4523-99a9-056d087aa290 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:35.290059 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e044f-48f3-4523-99a9-056d087aa290 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:35.290325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472e044f-48f3-4523-99a9-056d087aa290 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:35.293188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 890/1785] 10.222.0.17 () {54 vars in 921 bytes} [Sun Aug 7 19:26:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:26:35.487553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e53f0-dd41-49e3-984c-4b65e0c59c35 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:35.523473 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d7caf-fc70-433a-8e30-2e9791e9628d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:35.524749 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d7caf-fc70-433a-8e30-2e9791e9628d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:35.525484 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d7caf-fc70-433a-8e30-2e9791e9628d admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:35.525934 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d7caf-fc70-433a-8e30-2e9791e9628d admin admin] PATH_INFO: `/v3/users/d181ea0863e642668e437300cb41206e` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:35.530270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d7caf-fc70-433a-8e30-2e9791e9628d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d181ea0863e642668e437300cb41206e)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:35.535756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d7caf-fc70-433a-8e30-2e9791e9628d admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:35.599850 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b4c6b-be86-4843-907f-8e4b368e8734 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:35.600105 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b4c6b-be86-4843-907f-8e4b368e8734 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:35.600576 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b4c6b-be86-4843-907f-8e4b368e8734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:35.664930 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-679b4c6b-be86-4843-907f-8e4b368e8734 None None] MFA Rules not processed for user `a4946aec299f45bd8402cff3159d04d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:35.666286 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-679b4c6b-be86-4843-907f-8e4b368e8734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:35.685351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-355d7caf-fc70-433a-8e30-2e9791e9628d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd181ea0863e642668e437300cb41206e'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:35.715875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-355d7caf-fc70-433a-8e30-2e9791e9628d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd181ea0863e642668e437300cb41206e'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:35.756955 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-355d7caf-fc70-433a-8e30-2e9791e9628d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd181ea0863e642668e437300cb41206e'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:35.757798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-355d7caf-fc70-433a-8e30-2e9791e9628d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd181ea0863e642668e437300cb41206e'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:35.778449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-355d7caf-fc70-433a-8e30-2e9791e9628d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd181ea0863e642668e437300cb41206e'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:35.800038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 896/1786] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:35 2022] DELETE /identity/v3/users/d181ea0863e642668e437300cb41206e => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:26:35.804062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076d82d-1dff-4524-98e2-6a029bcc3ca1 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:35.805063 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076d82d-1dff-4524-98e2-6a029bcc3ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:35.806557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c076d82d-1dff-4524-98e2-6a029bcc3ca1 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:35.814662 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 897/1787] 10.222.0.17 () {54 vars in 921 bytes} [Sun Aug 7 19:26:35 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:26:35.841872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076d82d-1dff-4524-98e2-6a029bcc3ca1 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:35.845134 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c076d82d-1dff-4524-98e2-6a029bcc3ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:35.962251 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c076d82d-1dff-4524-98e2-6a029bcc3ca1 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:35.967488 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd805c-0f2c-4750-96c5-11cc1f014eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:35.968392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd805c-0f2c-4750-96c5-11cc1f014eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:35.968874 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd805c-0f2c-4750-96c5-11cc1f014eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:35.969284 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd805c-0f2c-4750-96c5-11cc1f014eb4 admin admin] PATH_INFO: `/v3/users/a0b2a4d397f14de2b8164761d03a1258` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:35.970901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd805c-0f2c-4750-96c5-11cc1f014eb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0b2a4d397f14de2b8164761d03a1258)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:35.974085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd805c-0f2c-4750-96c5-11cc1f014eb4 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:35.996094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 891/1788] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Aug 7 19:26:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:26:36.029261 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:36.043484 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b342e-0eaa-47e1-aca2-771bd2879471 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:36.043746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b342e-0eaa-47e1-aca2-771bd2879471 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:36.044054 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b342e-0eaa-47e1-aca2-771bd2879471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:36.077930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-9cdd805c-0f2c-4750-96c5-11cc1f014eb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0b2a4d397f14de2b8164761d03a1258'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:36.097264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-c27b342e-0eaa-47e1-aca2-771bd2879471 None None] MFA Rules not processed for user `a4946aec299f45bd8402cff3159d04d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:36.099364 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-9cdd805c-0f2c-4750-96c5-11cc1f014eb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0b2a4d397f14de2b8164761d03a1258'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:36.101216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-c27b342e-0eaa-47e1-aca2-771bd2879471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:36.126513 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-9cdd805c-0f2c-4750-96c5-11cc1f014eb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0b2a4d397f14de2b8164761d03a1258'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:36.127859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-9cdd805c-0f2c-4750-96c5-11cc1f014eb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0b2a4d397f14de2b8164761d03a1258'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:36.140945 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-9cdd805c-0f2c-4750-96c5-11cc1f014eb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0b2a4d397f14de2b8164761d03a1258'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:36.159604 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 898/1789] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:35 2022] DELETE /identity/v3/users/a0b2a4d397f14de2b8164761d03a1258 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:26:36.164061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec8722-dca5-4fd1-a7ed-07c7175e932d None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:36.164549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec8722-dca5-4fd1-a7ed-07c7175e932d None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:36.164984 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec8722-dca5-4fd1-a7ed-07c7175e932d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:36.300433 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-01ec8722-dca5-4fd1-a7ed-07c7175e932d None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:36.301929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-01ec8722-dca5-4fd1-a7ed-07c7175e932d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:36.378319 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 892/1790] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Aug 7 19:26:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:36.512048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 899/1791] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:36.774091 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b342e-0eaa-47e1-aca2-771bd2879471 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:36.776406 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-c27b342e-0eaa-47e1-aca2-771bd2879471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:36.958397 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b342e-0eaa-47e1-aca2-771bd2879471 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:37.004442 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:37.173172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe84f5-1c60-48a6-9f1d-999f58ada79d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:37.174252 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec8722-dca5-4fd1-a7ed-07c7175e932d None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:37.175079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe84f5-1c60-48a6-9f1d-999f58ada79d service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:37.175380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe84f5-1c60-48a6-9f1d-999f58ada79d service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:37.175656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe84f5-1c60-48a6-9f1d-999f58ada79d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:37.178300 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-01ec8722-dca5-4fd1-a7ed-07c7175e932d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:37.209931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe84f5-1c60-48a6-9f1d-999f58ada79d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:37.216043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe84f5-1c60-48a6-9f1d-999f58ada79d service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:37.255040 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 893/1792] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:37.447336 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec8722-dca5-4fd1-a7ed-07c7175e932d None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:37.466113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8943c-cae4-42ca-b052-4ea33e2968d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:37.467959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8943c-cae4-42ca-b052-4ea33e2968d1 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:37.468499 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8943c-cae4-42ca-b052-4ea33e2968d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:37.468965 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae8943c-cae4-42ca-b052-4ea33e2968d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:37.471754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4ae8943c-cae4-42ca-b052-4ea33e2968d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:37.491352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae8943c-cae4-42ca-b052-4ea33e2968d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:37.496342 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae8943c-cae4-42ca-b052-4ea33e2968d1 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:37.530746 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 900/1793] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:37.795695 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe84f5-1c60-48a6-9f1d-999f58ada79d service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:37.859308 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe84f5-1c60-48a6-9f1d-999f58ada79d service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:37.877591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890e012-6cf2-45ee-a4cc-3264942c17da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:37.880154 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890e012-6cf2-45ee-a4cc-3264942c17da service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:37.886808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890e012-6cf2-45ee-a4cc-3264942c17da service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:37.887408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890e012-6cf2-45ee-a4cc-3264942c17da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:37.926344 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890e012-6cf2-45ee-a4cc-3264942c17da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:37.928036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890e012-6cf2-45ee-a4cc-3264942c17da service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:37.972143 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 894/1794] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:38.678865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8943c-cae4-42ca-b052-4ea33e2968d1 service nova] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:38.680341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4ae8943c-cae4-42ca-b052-4ea33e2968d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:38.706936 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae8943c-cae4-42ca-b052-4ea33e2968d1 service nova] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:38.714416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2394971-9ebb-472f-9018-269c6a7ab460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:38.715070 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2394971-9ebb-472f-9018-269c6a7ab460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:38.715343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2394971-9ebb-472f-9018-269c6a7ab460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:38.715872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2394971-9ebb-472f-9018-269c6a7ab460 admin admin] PATH_INFO: `/v3/projects/8d5f0c98cce84b379ceac995b2d09e33` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:38.718134 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2394971-9ebb-472f-9018-269c6a7ab460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d5f0c98cce84b379ceac995b2d09e33)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:38.720815 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2394971-9ebb-472f-9018-269c6a7ab460 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:38.827488 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:38.910515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-f2394971-9ebb-472f-9018-269c6a7ab460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d5f0c98cce84b379ceac995b2d09e33', 'request_id': 'req-f2394971-9ebb-472f-9018-269c6a7ab460'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:38.927456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-f2394971-9ebb-472f-9018-269c6a7ab460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d5f0c98cce84b379ceac995b2d09e33', 'request_id': 'req-f2394971-9ebb-472f-9018-269c6a7ab460'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:38.944433 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-f2394971-9ebb-472f-9018-269c6a7ab460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d5f0c98cce84b379ceac995b2d09e33', 'request_id': 'req-f2394971-9ebb-472f-9018-269c6a7ab460'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:38.980204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-f2394971-9ebb-472f-9018-269c6a7ab460 admin admin] The token cache is being invalidate because project 8d5f0c98cce84b379ceac995b2d09e33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:38.981517 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-f2394971-9ebb-472f-9018-269c6a7ab460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:38.983439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-f2394971-9ebb-472f-9018-269c6a7ab460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:38.987307 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 901/1795] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:38 2022] DELETE /identity/v3/projects/8d5f0c98cce84b379ceac995b2d09e33 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:38.991772 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890e012-6cf2-45ee-a4cc-3264942c17da service nova] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:39.004472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890e012-6cf2-45ee-a4cc-3264942c17da service nova] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:39.140174 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f5f13-de7d-48f5-a36a-d9c4a3933c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:39.141158 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f5f13-de7d-48f5-a36a-d9c4a3933c53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:39.142328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f5f13-de7d-48f5-a36a-d9c4a3933c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:39.142495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f5f13-de7d-48f5-a36a-d9c4a3933c53 admin admin] PATH_INFO: `/v3/users/87602e0463c94d50821ca952871aaaa2` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:39.144290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f5f13-de7d-48f5-a36a-d9c4a3933c53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87602e0463c94d50821ca952871aaaa2)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:39.146675 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f5f13-de7d-48f5-a36a-d9c4a3933c53 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:39.238604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-890f5f13-de7d-48f5-a36a-d9c4a3933c53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87602e0463c94d50821ca952871aaaa2'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:39.263895 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-890f5f13-de7d-48f5-a36a-d9c4a3933c53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87602e0463c94d50821ca952871aaaa2'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:39.274925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-890f5f13-de7d-48f5-a36a-d9c4a3933c53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87602e0463c94d50821ca952871aaaa2'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:39.276113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-890f5f13-de7d-48f5-a36a-d9c4a3933c53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87602e0463c94d50821ca952871aaaa2'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:39.277039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-890f5f13-de7d-48f5-a36a-d9c4a3933c53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87602e0463c94d50821ca952871aaaa2'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:39.306052 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 895/1796] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:38 2022] DELETE /identity/v3/users/87602e0463c94d50821ca952871aaaa2 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:39.712773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2394971-9ebb-472f-9018-269c6a7ab460 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:39.715740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f2394971-9ebb-472f-9018-269c6a7ab460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:39.743446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2394971-9ebb-472f-9018-269c6a7ab460 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:39.761417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f5f13-de7d-48f5-a36a-d9c4a3933c53 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:39.763186 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-890f5f13-de7d-48f5-a36a-d9c4a3933c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:39.832463 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:39.872318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:39.872990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:39.873270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:39.873541 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf admin admin] PATH_INFO: `/v3/projects/d6007723a068405d9032c43177fe5097` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:39.876472 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6007723a068405d9032c43177fe5097)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:39.883401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:39.951390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f5f13-de7d-48f5-a36a-d9c4a3933c53 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:39.956466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132445a5-c62e-4817-b2c8-8af458a9850a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:39.957308 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132445a5-c62e-4817-b2c8-8af458a9850a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:39.957600 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132445a5-c62e-4817-b2c8-8af458a9850a admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:39.957873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132445a5-c62e-4817-b2c8-8af458a9850a admin admin] PATH_INFO: `/v3/projects/02d5eeb719a64074a7ab459e2652fd28` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:39.959606 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132445a5-c62e-4817-b2c8-8af458a9850a admin admin] RBAC: Authorizing `identity:delete_project(project_id=02d5eeb719a64074a7ab459e2652fd28)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:39.961829 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132445a5-c62e-4817-b2c8-8af458a9850a admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:39.998725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6007723a068405d9032c43177fe5097', 'request_id': 'req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:40.007583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6007723a068405d9032c43177fe5097', 'request_id': 'req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:40.017691 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6007723a068405d9032c43177fe5097', 'request_id': 'req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:40.046147 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf admin admin] The token cache is being invalidate because project d6007723a068405d9032c43177fe5097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:40.046887 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:40.048528 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:40.050174 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-132445a5-c62e-4817-b2c8-8af458a9850a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02d5eeb719a64074a7ab459e2652fd28', 'request_id': 'req-132445a5-c62e-4817-b2c8-8af458a9850a'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:40.058691 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 902/1797] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:39 2022] DELETE /identity/v3/projects/d6007723a068405d9032c43177fe5097 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:40.059486 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:40.062145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:40.067591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-132445a5-c62e-4817-b2c8-8af458a9850a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02d5eeb719a64074a7ab459e2652fd28', 'request_id': 'req-132445a5-c62e-4817-b2c8-8af458a9850a'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:40.081272 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-132445a5-c62e-4817-b2c8-8af458a9850a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02d5eeb719a64074a7ab459e2652fd28', 'request_id': 'req-132445a5-c62e-4817-b2c8-8af458a9850a'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:40.100961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-132445a5-c62e-4817-b2c8-8af458a9850a admin admin] The token cache is being invalidate because project 02d5eeb719a64074a7ab459e2652fd28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:40.100961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-132445a5-c62e-4817-b2c8-8af458a9850a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:40.102554 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-132445a5-c62e-4817-b2c8-8af458a9850a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:40.108328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 896/1798] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:39 2022] DELETE /identity/v3/projects/02d5eeb719a64074a7ab459e2652fd28 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:40.110028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d017a-a1b2-4674-8fa4-cc018c4ed4fa None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.110320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d017a-a1b2-4674-8fa4-cc018c4ed4fa None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.110635 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d017a-a1b2-4674-8fa4-cc018c4ed4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.136530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-a95d017a-a1b2-4674-8fa4-cc018c4ed4fa None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:40.137773 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-a95d017a-a1b2-4674-8fa4-cc018c4ed4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:40.175529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:40.199440 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fdbcc-d97f-4096-aa5b-7f47f95a39cf admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:40.236657 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 897/1799] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:40.238992 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3a5ab-7e57-4aff-a186-e368b95655b3 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.239342 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3a5ab-7e57-4aff-a186-e368b95655b3 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.239644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3a5ab-7e57-4aff-a186-e368b95655b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.268807 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-49f3a5ab-7e57-4aff-a186-e368b95655b3 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:40.270701 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-49f3a5ab-7e57-4aff-a186-e368b95655b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:40.280303 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 898/1800] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:40.281814 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a0a64-4a3a-455c-8288-a309ef6e4222 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.282009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a0a64-4a3a-455c-8288-a309ef6e4222 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.282314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a0a64-4a3a-455c-8288-a309ef6e4222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.316238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-424a0a64-4a3a-455c-8288-a309ef6e4222 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:40.317585 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-424a0a64-4a3a-455c-8288-a309ef6e4222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:40.322738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:40.323454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.323744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.324063 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33 admin admin] PATH_INFO: `/v3/projects/c992244de13c4f0bb6244e4f9ea53001` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.325450 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c992244de13c4f0bb6244e4f9ea53001)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:40.327097 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 899/1801] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:40.330586 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:40.332452 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca4c0f-6178-4f40-adce-f4755bb84ca0 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.332944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca4c0f-6178-4f40-adce-f4755bb84ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.333404 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca4c0f-6178-4f40-adce-f4755bb84ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.353817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-e2ca4c0f-6178-4f40-adce-f4755bb84ca0 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:40.355146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e2ca4c0f-6178-4f40-adce-f4755bb84ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:40.364643 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 900/1802] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:40.368405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca4c0f-6178-4f40-adce-f4755bb84ca0 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:40.382334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca4c0f-6178-4f40-adce-f4755bb84ca0 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:40.388772 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09979f-1b4a-47bf-9b50-070cb50c2a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:40.389548 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09979f-1b4a-47bf-9b50-070cb50c2a3b admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.389868 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09979f-1b4a-47bf-9b50-070cb50c2a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.390211 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09979f-1b4a-47bf-9b50-070cb50c2a3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.391316 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09979f-1b4a-47bf-9b50-070cb50c2a3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:26:40.391632 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09979f-1b4a-47bf-9b50-070cb50c2a3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:40.393567 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09979f-1b4a-47bf-9b50-070cb50c2a3b admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:40.412325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 901/1803] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:26:40.412325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09979f-1b4a-47bf-9b50-070cb50c2a3b admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:40.426173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09979f-1b4a-47bf-9b50-070cb50c2a3b admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:40.427345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c992244de13c4f0bb6244e4f9ea53001', 'request_id': 'req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:40.435290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c992244de13c4f0bb6244e4f9ea53001', 'request_id': 'req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:40.442989 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c992244de13c4f0bb6244e4f9ea53001', 'request_id': 'req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:40.447453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:40.464808 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33 admin admin] The token cache is being invalidate because project c992244de13c4f0bb6244e4f9ea53001 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:40.465152 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:40.465976 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:40.471186 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 903/1804] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:40 2022] DELETE /identity/v3/projects/c992244de13c4f0bb6244e4f9ea53001 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:40.471186 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:40.472635 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:40.542676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297dd961-8afc-4f68-9d87-479e6d0b1982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:40.543429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dd961-8afc-4f68-9d87-479e6d0b1982 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.543968 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dd961-8afc-4f68-9d87-479e6d0b1982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.544171 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dd961-8afc-4f68-9d87-479e6d0b1982 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.544828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297dd961-8afc-4f68-9d87-479e6d0b1982 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:26:40.546740 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297dd961-8afc-4f68-9d87-479e6d0b1982 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:40.550715 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297dd961-8afc-4f68-9d87-479e6d0b1982 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:40.572327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 902/1805] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:26:40.572327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c518f-d2ae-481c-babe-b698082175b7 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.572327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c518f-d2ae-481c-babe-b698082175b7 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.572327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c518f-d2ae-481c-babe-b698082175b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.572760 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27cdee-de03-46e6-a9b2-dc518e4f6d33 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:40.577323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d7926-d609-415e-a479-1eb778cc3cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:40.578082 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d7926-d609-415e-a479-1eb778cc3cfb admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.578391 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d7926-d609-415e-a479-1eb778cc3cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.578741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d7926-d609-415e-a479-1eb778cc3cfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.579424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d7926-d609-415e-a479-1eb778cc3cfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:40.581569 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d7926-d609-415e-a479-1eb778cc3cfb admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:40.592870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-f90c518f-d2ae-481c-babe-b698082175b7 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:40.594512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f90c518f-d2ae-481c-babe-b698082175b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:40.605499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 903/1806] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:40.607317 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c518f-d2ae-481c-babe-b698082175b7 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:40.610338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f90c518f-d2ae-481c-babe-b698082175b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:40.618381 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 904/1807] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:40 2022] POST /identity/v3/projects => generated 399 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:40.620246 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c5bbb-10af-4030-a53b-3effada2c5ae None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.620246 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c5bbb-10af-4030-a53b-3effada2c5ae None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.620401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c5bbb-10af-4030-a53b-3effada2c5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.633080 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c518f-d2ae-481c-babe-b698082175b7 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:40.644783 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-de3c5bbb-10af-4030-a53b-3effada2c5ae None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:40.646022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-de3c5bbb-10af-4030-a53b-3effada2c5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:40.652017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:40.723498 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a48051-8eb0-45f0-a6f4-5ba1c9c79c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:40.724171 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a48051-8eb0-45f0-a6f4-5ba1c9c79c8c admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.724456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a48051-8eb0-45f0-a6f4-5ba1c9c79c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.724739 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a48051-8eb0-45f0-a6f4-5ba1c9c79c8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.725396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a48051-8eb0-45f0-a6f4-5ba1c9c79c8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:40.727319 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a48051-8eb0-45f0-a6f4-5ba1c9c79c8c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:40.747934 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 905/1808] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:40.749665 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c5bbb-10af-4030-a53b-3effada2c5ae None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:40.762620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c5bbb-10af-4030-a53b-3effada2c5ae None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:40.771100 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8a54e-3f1a-4343-a80c-269b2f886133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:40.776747 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8a54e-3f1a-4343-a80c-269b2f886133 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.779190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8a54e-3f1a-4343-a80c-269b2f886133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.779190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8a54e-3f1a-4343-a80c-269b2f886133 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.780388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 904/1809] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:40 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:40.780954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f8a54e-3f1a-4343-a80c-269b2f886133 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:40.781334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a48051-8eb0-45f0-a6f4-5ba1c9c79c8c admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:40.781688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f8a54e-3f1a-4343-a80c-269b2f886133 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:40.812910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 906/1810] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:40 2022] POST /identity/v3/users => generated 300 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:40.812910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8a54e-3f1a-4343-a80c-269b2f886133 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:40.821079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8a54e-3f1a-4343-a80c-269b2f886133 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:40.884524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19219d-1197-4d6d-a601-0e70749c7476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:40.885000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a48051-8eb0-45f0-a6f4-5ba1c9c79c8c admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:40.885151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19219d-1197-4d6d-a601-0e70749c7476 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.885437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19219d-1197-4d6d-a601-0e70749c7476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.885709 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19219d-1197-4d6d-a601-0e70749c7476 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.886344 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19219d-1197-4d6d-a601-0e70749c7476 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:40.888326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19219d-1197-4d6d-a601-0e70749c7476 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:40.893566 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7acebb-16ee-437b-9bdc-b6a6e3be4822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:40.894671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7acebb-16ee-437b-9bdc-b6a6e3be4822 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.895198 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7acebb-16ee-437b-9bdc-b6a6e3be4822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.895722 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7acebb-16ee-437b-9bdc-b6a6e3be4822 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.897017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7acebb-16ee-437b-9bdc-b6a6e3be4822 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:26:40.897643 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7acebb-16ee-437b-9bdc-b6a6e3be4822 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:40.901179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7acebb-16ee-437b-9bdc-b6a6e3be4822 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:40.917196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 907/1811] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:40 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:40.917196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19219d-1197-4d6d-a601-0e70749c7476 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:40.926217 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19219d-1197-4d6d-a601-0e70749c7476 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:40.927135 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 905/1812] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:26:40.928307 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7acebb-16ee-437b-9bdc-b6a6e3be4822 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:40.934797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272b41d-b19e-4198-a42a-cdab2e024d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:40.935434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272b41d-b19e-4198-a42a-cdab2e024d5d admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.935725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272b41d-b19e-4198-a42a-cdab2e024d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.936043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272b41d-b19e-4198-a42a-cdab2e024d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.936698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8272b41d-b19e-4198-a42a-cdab2e024d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:40.938359 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8272b41d-b19e-4198-a42a-cdab2e024d5d admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:40.941492 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7acebb-16ee-437b-9bdc-b6a6e3be4822 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:40.946366 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53369262-67bc-4b3c-b12b-f1bfce656502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:40.946990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53369262-67bc-4b3c-b12b-f1bfce656502 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.947263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53369262-67bc-4b3c-b12b-f1bfce656502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.947525 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53369262-67bc-4b3c-b12b-f1bfce656502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.948208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53369262-67bc-4b3c-b12b-f1bfce656502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:40.950139 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53369262-67bc-4b3c-b12b-f1bfce656502 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:40.950273 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 908/1813] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:40.950865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272b41d-b19e-4198-a42a-cdab2e024d5d admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:40.963855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 906/1814] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:40.964880 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53369262-67bc-4b3c-b12b-f1bfce656502 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:40.965060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272b41d-b19e-4198-a42a-cdab2e024d5d admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:40.973477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3d4cf-8261-4996-b732-a2b5294d7f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:40.974333 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d4cf-8261-4996-b732-a2b5294d7f41 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.974653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d4cf-8261-4996-b732-a2b5294d7f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.974964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3d4cf-8261-4996-b732-a2b5294d7f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.975489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53369262-67bc-4b3c-b12b-f1bfce656502 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:40.975646 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3d4cf-8261-4996-b732-a2b5294d7f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:40.978035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3d4cf-8261-4996-b732-a2b5294d7f41 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:40.980537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cdba27-c39d-48c9-9e6d-c17aea9f3e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:40.981282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdba27-c39d-48c9-9e6d-c17aea9f3e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:40.981468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdba27-c39d-48c9-9e6d-c17aea9f3e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:40.981698 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdba27-c39d-48c9-9e6d-c17aea9f3e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:40.982500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cdba27-c39d-48c9-9e6d-c17aea9f3e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:40.984120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cdba27-c39d-48c9-9e6d-c17aea9f3e3d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:40.994832 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 907/1815] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:40.995685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cdba27-c39d-48c9-9e6d-c17aea9f3e3d admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:41.001242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:41.008364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cdba27-c39d-48c9-9e6d-c17aea9f3e3d admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:41.015578 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa12f4-3171-42c5-9d15-c873d1e6d335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:41.016240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa12f4-3171-42c5-9d15-c873d1e6d335 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:41.016518 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa12f4-3171-42c5-9d15-c873d1e6d335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:41.016903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa12f4-3171-42c5-9d15-c873d1e6d335 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:41.017592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa12f4-3171-42c5-9d15-c873d1e6d335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:41.018451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 909/1816] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:40 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:41.019200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa12f4-3171-42c5-9d15-c873d1e6d335 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:41.019705 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3d4cf-8261-4996-b732-a2b5294d7f41 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:41.031304 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 908/1817] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:41.032185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa12f4-3171-42c5-9d15-c873d1e6d335 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:41.043826 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa12f4-3171-42c5-9d15-c873d1e6d335 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:41.084019 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:41.104093 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3d4cf-8261-4996-b732-a2b5294d7f41 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:41.109296 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e425e30-4b3f-42f0-a556-4a5ff6947078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:41.110127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e425e30-4b3f-42f0-a556-4a5ff6947078 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:41.110455 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e425e30-4b3f-42f0-a556-4a5ff6947078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:41.110786 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e425e30-4b3f-42f0-a556-4a5ff6947078 admin admin] PATH_INFO: `/v3/projects/66c4e94b6b354f60984ec37dff8b6af5/users/c1361321974f4af080f4618a5ed1c8a7/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:41.131495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e425e30-4b3f-42f0-a556-4a5ff6947078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66c4e94b6b354f60984ec37dff8b6af5, user_id=c1361321974f4af080f4618a5ed1c8a7, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:41.136997 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e425e30-4b3f-42f0-a556-4a5ff6947078 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:41.150781 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc7c5b0-a250-4a9e-9436-16fb29c6a428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:41.151402 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7c5b0-a250-4a9e-9436-16fb29c6a428 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:41.151706 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7c5b0-a250-4a9e-9436-16fb29c6a428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:41.151981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc7c5b0-a250-4a9e-9436-16fb29c6a428 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:41.152666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7c5b0-a250-4a9e-9436-16fb29c6a428 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:41.156055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc7c5b0-a250-4a9e-9436-16fb29c6a428 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:41.169022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 910/1818] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:41 2022] PUT /identity/v3/projects/66c4e94b6b354f60984ec37dff8b6af5/users/c1361321974f4af080f4618a5ed1c8a7/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:41.170423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e425e30-4b3f-42f0-a556-4a5ff6947078 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:41.188561 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 909/1819] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:41 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:41.190184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff017fa-33f0-482e-93e2-4e1b8ad51b30 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:41.193142 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff017fa-33f0-482e-93e2-4e1b8ad51b30 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:41.193142 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff017fa-33f0-482e-93e2-4e1b8ad51b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:41.238996 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-aff017fa-33f0-482e-93e2-4e1b8ad51b30 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:41.240584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-aff017fa-33f0-482e-93e2-4e1b8ad51b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:41.322680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e425e30-4b3f-42f0-a556-4a5ff6947078 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:41.347876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f2340b-9a27-4c6d-be96-5863a09b7420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:41.348707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2340b-9a27-4c6d-be96-5863a09b7420 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:41.348983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2340b-9a27-4c6d-be96-5863a09b7420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:41.349265 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2340b-9a27-4c6d-be96-5863a09b7420 admin admin] PATH_INFO: `/v3/projects/076249e805944633bc76d15cc9eaff2b/users/1791564694ba4784b6f7d59470ac426a/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:41.382439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2340b-9a27-4c6d-be96-5863a09b7420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=076249e805944633bc76d15cc9eaff2b, user_id=1791564694ba4784b6f7d59470ac426a, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:41.385333 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f2340b-9a27-4c6d-be96-5863a09b7420 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:41.404046 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 910/1820] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:41.404347 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 911/1821] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:41 2022] PUT /identity/v3/projects/076249e805944633bc76d15cc9eaff2b/users/1791564694ba4784b6f7d59470ac426a/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:41.406281 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff017fa-33f0-482e-93e2-4e1b8ad51b30 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:41.415808 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155eda9-96ff-4b99-9152-95d6dad76341 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:41.416155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155eda9-96ff-4b99-9152-95d6dad76341 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:41.416438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155eda9-96ff-4b99-9152-95d6dad76341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:41.429719 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff017fa-33f0-482e-93e2-4e1b8ad51b30 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:41.458372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-a155eda9-96ff-4b99-9152-95d6dad76341 None None] MFA Rules not processed for user `1791564694ba4784b6f7d59470ac426a`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:41.459677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a155eda9-96ff-4b99-9152-95d6dad76341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:41.538735 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cb7a9-a159-4b66-acbd-efa8cb7ad8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:41.539414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cb7a9-a159-4b66-acbd-efa8cb7ad8f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:41.539696 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cb7a9-a159-4b66-acbd-efa8cb7ad8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:41.539967 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cb7a9-a159-4b66-acbd-efa8cb7ad8f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:41.540666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994cb7a9-a159-4b66-acbd-efa8cb7ad8f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:41.542370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994cb7a9-a159-4b66-acbd-efa8cb7ad8f9 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:41.554144 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 911/1822] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:41.555683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cb7a9-a159-4b66-acbd-efa8cb7ad8f9 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:41.557432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-994cb7a9-a159-4b66-acbd-efa8cb7ad8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:41.564456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 912/1823] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:41.569367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155eda9-96ff-4b99-9152-95d6dad76341 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:41.581390 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155eda9-96ff-4b99-9152-95d6dad76341 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:41.583237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:41.586988 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6404e0d-edb5-4bd5-a4bd-5d4775505513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:41.587640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6404e0d-edb5-4bd5-a4bd-5d4775505513 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:41.588040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6404e0d-edb5-4bd5-a4bd-5d4775505513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:41.588323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6404e0d-edb5-4bd5-a4bd-5d4775505513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:41.589369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6404e0d-edb5-4bd5-a4bd-5d4775505513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:41.591692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6404e0d-edb5-4bd5-a4bd-5d4775505513 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:41.604195 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 913/1824] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:41.605875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6404e0d-edb5-4bd5-a4bd-5d4775505513 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:41.609000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c6404e0d-edb5-4bd5-a4bd-5d4775505513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:41.628610 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cb7a9-a159-4b66-acbd-efa8cb7ad8f9 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:41.633036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda808a-99cc-4376-beb5-53f3ff5398d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:41.633676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda808a-99cc-4376-beb5-53f3ff5398d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:41.633958 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda808a-99cc-4376-beb5-53f3ff5398d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:41.634227 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adda808a-99cc-4376-beb5-53f3ff5398d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:41.644549 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda808a-99cc-4376-beb5-53f3ff5398d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:41.646045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adda808a-99cc-4376-beb5-53f3ff5398d4 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:41.649074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:41.657029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 912/1825] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:41.657789 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda808a-99cc-4376-beb5-53f3ff5398d4 service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:41.667716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adda808a-99cc-4376-beb5-53f3ff5398d4 service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:41.674543 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a313602-f560-4b06-b457-53306de09d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:41.675254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a313602-f560-4b06-b457-53306de09d35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:41.675531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a313602-f560-4b06-b457-53306de09d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:41.675785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a313602-f560-4b06-b457-53306de09d35 admin admin] PATH_INFO: `/v3/projects/91eaae76dd3e43b5817ef44f63fe0329/users/e218c2d173d4466ea8da29bb2371fec1/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:41.677970 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6404e0d-edb5-4bd5-a4bd-5d4775505513 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:41.690186 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0e646-84bc-477c-bcf0-ad87dc030982 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:41.690830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0e646-84bc-477c-bcf0-ad87dc030982 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:41.691135 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0e646-84bc-477c-bcf0-ad87dc030982 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:41.691363 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0e646-84bc-477c-bcf0-ad87dc030982 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:41.700760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a313602-f560-4b06-b457-53306de09d35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91eaae76dd3e43b5817ef44f63fe0329, user_id=e218c2d173d4466ea8da29bb2371fec1, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:41.702036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0e646-84bc-477c-bcf0-ad87dc030982 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:41.703572 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e0e646-84bc-477c-bcf0-ad87dc030982 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:41.706475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a313602-f560-4b06-b457-53306de09d35 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:41.732206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 914/1826] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:41.740620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0e646-84bc-477c-bcf0-ad87dc030982 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:41.753679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0e646-84bc-477c-bcf0-ad87dc030982 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:41.760245 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 913/1827] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:41 2022] PUT /identity/v3/projects/91eaae76dd3e43b5817ef44f63fe0329/users/e218c2d173d4466ea8da29bb2371fec1/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:41.766149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92d04d-c76e-44de-9fcf-fe88023ea965 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:41.766524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92d04d-c76e-44de-9fcf-fe88023ea965 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:41.767746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92d04d-c76e-44de-9fcf-fe88023ea965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:41.819040 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-aa92d04d-c76e-44de-9fcf-fe88023ea965 None None] MFA Rules not processed for user `e218c2d173d4466ea8da29bb2371fec1`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:41.820680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-aa92d04d-c76e-44de-9fcf-fe88023ea965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:41.880778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0d8f8-6fd0-40ed-8f24-6dd6afdad776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:41.881967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0d8f8-6fd0-40ed-8f24-6dd6afdad776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:41.882390 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0d8f8-6fd0-40ed-8f24-6dd6afdad776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:41.882802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0d8f8-6fd0-40ed-8f24-6dd6afdad776 admin admin] PATH_INFO: `/v3/users/1791564694ba4784b6f7d59470ac426a` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:41.884314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f0d8f8-6fd0-40ed-8f24-6dd6afdad776 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1791564694ba4784b6f7d59470ac426a)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:41.886750 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f0d8f8-6fd0-40ed-8f24-6dd6afdad776 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:41.908993 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 914/1828] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:41 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:26:41.910844 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92d04d-c76e-44de-9fcf-fe88023ea965 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:41.925468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92d04d-c76e-44de-9fcf-fe88023ea965 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:41.932140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731eccd8-8437-4239-a63c-cb09a4a8b0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:41.932957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731eccd8-8437-4239-a63c-cb09a4a8b0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:41.933387 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731eccd8-8437-4239-a63c-cb09a4a8b0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:41.933633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731eccd8-8437-4239-a63c-cb09a4a8b0e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:41.934498 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731eccd8-8437-4239-a63c-cb09a4a8b0e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:41.936659 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731eccd8-8437-4239-a63c-cb09a4a8b0e8 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:41.958073 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-66f0d8f8-6fd0-40ed-8f24-6dd6afdad776 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1791564694ba4784b6f7d59470ac426a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:41.969443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-66f0d8f8-6fd0-40ed-8f24-6dd6afdad776 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1791564694ba4784b6f7d59470ac426a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:42.005102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-66f0d8f8-6fd0-40ed-8f24-6dd6afdad776 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1791564694ba4784b6f7d59470ac426a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:42.005954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-66f0d8f8-6fd0-40ed-8f24-6dd6afdad776 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1791564694ba4784b6f7d59470ac426a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:42.018033 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-66f0d8f8-6fd0-40ed-8f24-6dd6afdad776 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1791564694ba4784b6f7d59470ac426a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:42.024727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 915/1829] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:41 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:42.025819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731eccd8-8437-4239-a63c-cb09a4a8b0e8 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:42.027530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-731eccd8-8437-4239-a63c-cb09a4a8b0e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:42.048087 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 915/1830] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:41 2022] DELETE /identity/v3/users/1791564694ba4784b6f7d59470ac426a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:42.054176 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9e8a3-68b4-4a6a-b78d-1c9da71b2b50 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:42.054677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9e8a3-68b4-4a6a-b78d-1c9da71b2b50 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:42.055146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9e8a3-68b4-4a6a-b78d-1c9da71b2b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:42.093136 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-d1b9e8a3-68b4-4a6a-b78d-1c9da71b2b50 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:42.095637 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-d1b9e8a3-68b4-4a6a-b78d-1c9da71b2b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:42.158383 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731eccd8-8437-4239-a63c-cb09a4a8b0e8 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:42.182286 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:42.200423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:42.226556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 916/1831] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:26:42.245975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9e8a3-68b4-4a6a-b78d-1c9da71b2b50 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:42.247465 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-d1b9e8a3-68b4-4a6a-b78d-1c9da71b2b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:42.253653 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f7c14b-031b-4e4d-b0f2-edaa38294ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:42.254686 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7c14b-031b-4e4d-b0f2-edaa38294ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:42.255107 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7c14b-031b-4e4d-b0f2-edaa38294ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:42.255517 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f7c14b-031b-4e4d-b0f2-edaa38294ee9 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:42.256540 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f7c14b-031b-4e4d-b0f2-edaa38294ee9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:42.258667 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f7c14b-031b-4e4d-b0f2-edaa38294ee9 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:42.293258 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 916/1832] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:42 2022] POST /identity/v3/users => generated 294 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:42.302740 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f7c14b-031b-4e4d-b0f2-edaa38294ee9 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:42.316643 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f7c14b-031b-4e4d-b0f2-edaa38294ee9 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:42.324091 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878d609-a313-403a-a181-60a5140c7a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:42.325149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878d609-a313-403a-a181-60a5140c7a4c admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:42.325149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878d609-a313-403a-a181-60a5140c7a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:42.325149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878d609-a313-403a-a181-60a5140c7a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:42.325940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878d609-a313-403a-a181-60a5140c7a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:42.327849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878d609-a313-403a-a181-60a5140c7a4c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:42.336331 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9e8a3-68b4-4a6a-b78d-1c9da71b2b50 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:42.342247 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f1b4a-54bf-43dc-aae4-7b5170422334 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:42.343172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f1b4a-54bf-43dc-aae4-7b5170422334 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:42.343537 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f1b4a-54bf-43dc-aae4-7b5170422334 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:42.343906 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f1b4a-54bf-43dc-aae4-7b5170422334 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:42.346290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 917/1833] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:26:42.350814 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878d609-a313-403a-a181-60a5140c7a4c admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:42.363314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878d609-a313-403a-a181-60a5140c7a4c admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:42.366124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f1b4a-54bf-43dc-aae4-7b5170422334 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:42.367664 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f1b4a-54bf-43dc-aae4-7b5170422334 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:42.375535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc684660-2ccf-41b1-a200-0f24c3c21023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:42.377331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc684660-2ccf-41b1-a200-0f24c3c21023 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:42.378066 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc684660-2ccf-41b1-a200-0f24c3c21023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:42.378416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc684660-2ccf-41b1-a200-0f24c3c21023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:42.380833 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc684660-2ccf-41b1-a200-0f24c3c21023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:42.386846 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc684660-2ccf-41b1-a200-0f24c3c21023 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:42.389523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 917/1834] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:26:42.417467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f1b4a-54bf-43dc-aae4-7b5170422334 service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:42.425849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 918/1835] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:26:42.430661 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f1b4a-54bf-43dc-aae4-7b5170422334 service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:42.436393 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238932fb-65fe-4682-8e80-1e0eadda3445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:42.437422 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238932fb-65fe-4682-8e80-1e0eadda3445 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:42.437772 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238932fb-65fe-4682-8e80-1e0eadda3445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:42.438249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238932fb-65fe-4682-8e80-1e0eadda3445 admin admin] PATH_INFO: `/v3/projects/1bb884d16ca645c39918e9c6739e4e92/users/d5b987197e8a491a8207a9b521bef86a/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:42.502269 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238932fb-65fe-4682-8e80-1e0eadda3445 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb884d16ca645c39918e9c6739e4e92, user_id=d5b987197e8a491a8207a9b521bef86a, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:42.529401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238932fb-65fe-4682-8e80-1e0eadda3445 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:42.604895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 918/1836] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:42 2022] PUT /identity/v3/projects/1bb884d16ca645c39918e9c6739e4e92/users/d5b987197e8a491a8207a9b521bef86a/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:42.628640 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9117bd-53d6-49bb-9c28-cc7dcda67dc6 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:42.629176 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9117bd-53d6-49bb-9c28-cc7dcda67dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:42.629830 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9117bd-53d6-49bb-9c28-cc7dcda67dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:42.702465 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-db9117bd-53d6-49bb-9c28-cc7dcda67dc6 None None] MFA Rules not processed for user `d5b987197e8a491a8207a9b521bef86a`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:42.705727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-db9117bd-53d6-49bb-9c28-cc7dcda67dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:42.892271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 919/1837] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:42.897162 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238932fb-65fe-4682-8e80-1e0eadda3445 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:43.054822 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238932fb-65fe-4682-8e80-1e0eadda3445 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:43.067597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68c936-7614-4d8c-8bb6-0cbef7ad73e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:43.068428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68c936-7614-4d8c-8bb6-0cbef7ad73e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:43.068822 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68c936-7614-4d8c-8bb6-0cbef7ad73e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:43.069222 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68c936-7614-4d8c-8bb6-0cbef7ad73e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:43.069955 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae68c936-7614-4d8c-8bb6-0cbef7ad73e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:43.072112 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae68c936-7614-4d8c-8bb6-0cbef7ad73e9 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:43.143605 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9117bd-53d6-49bb-9c28-cc7dcda67dc6 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:43.144688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 919/1838] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:42 2022] POST /identity/v3/projects => generated 385 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:43.166970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9117bd-53d6-49bb-9c28-cc7dcda67dc6 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:43.227539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68c936-7614-4d8c-8bb6-0cbef7ad73e9 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:43.232043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-ae68c936-7614-4d8c-8bb6-0cbef7ad73e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:43.327714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed7736-7aba-4f8a-88e4-c84505802eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:43.328393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed7736-7aba-4f8a-88e4-c84505802eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:43.328673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed7736-7aba-4f8a-88e4-c84505802eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:43.328947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed7736-7aba-4f8a-88e4-c84505802eb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:43.329888 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed7736-7aba-4f8a-88e4-c84505802eb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:43.332456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed7736-7aba-4f8a-88e4-c84505802eb8 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:43.381841 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:43.397479 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 920/1839] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:43 2022] POST /identity/v3/users => generated 299 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:26:43.399625 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed7736-7aba-4f8a-88e4-c84505802eb8 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:43.421607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed7736-7aba-4f8a-88e4-c84505802eb8 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:43.432057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68c936-7614-4d8c-8bb6-0cbef7ad73e9 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:43.434080 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76decee-3398-4e27-8b55-941ed74de24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:43.435435 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76decee-3398-4e27-8b55-941ed74de24c admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:43.435953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76decee-3398-4e27-8b55-941ed74de24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:43.436496 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76decee-3398-4e27-8b55-941ed74de24c admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:43.437972 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76decee-3398-4e27-8b55-941ed74de24c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:43.441116 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a382b92-bdd9-4aed-b607-b751910552bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:43.441704 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76decee-3398-4e27-8b55-941ed74de24c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:43.442661 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a382b92-bdd9-4aed-b607-b751910552bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:43.443353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a382b92-bdd9-4aed-b607-b751910552bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:43.443792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a382b92-bdd9-4aed-b607-b751910552bf admin admin] PATH_INFO: `/v3/projects/076249e805944633bc76d15cc9eaff2b` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:43.457128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a382b92-bdd9-4aed-b607-b751910552bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=076249e805944633bc76d15cc9eaff2b)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:43.461166 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a382b92-bdd9-4aed-b607-b751910552bf admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:43.463400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:43.469651 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 921/1840] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:26:43.474960 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76decee-3398-4e27-8b55-941ed74de24c admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:43.496012 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76decee-3398-4e27-8b55-941ed74de24c admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:43.503308 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73e9cc-6adc-441c-b079-cfc79551bfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:43.504370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73e9cc-6adc-441c-b079-cfc79551bfcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:43.504823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73e9cc-6adc-441c-b079-cfc79551bfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:43.505329 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73e9cc-6adc-441c-b079-cfc79551bfcb admin admin] PATH_INFO: `/v3/projects/ce4d3f3bce174b3e924d517763b02a9a/users/4e8fb30a505144ad8685cca92446269a/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:43.532281 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73e9cc-6adc-441c-b079-cfc79551bfcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce4d3f3bce174b3e924d517763b02a9a, user_id=4e8fb30a505144ad8685cca92446269a, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:43.535216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c73e9cc-6adc-441c-b079-cfc79551bfcb admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:43.562963 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 922/1841] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:43 2022] PUT /identity/v3/projects/ce4d3f3bce174b3e924d517763b02a9a/users/4e8fb30a505144ad8685cca92446269a/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:43.568785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73e9cc-6adc-441c-b079-cfc79551bfcb admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:43.570702 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7a382b92-bdd9-4aed-b607-b751910552bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '076249e805944633bc76d15cc9eaff2b', 'request_id': 'req-7a382b92-bdd9-4aed-b607-b751910552bf'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:43.593144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7a382b92-bdd9-4aed-b607-b751910552bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '076249e805944633bc76d15cc9eaff2b', 'request_id': 'req-7a382b92-bdd9-4aed-b607-b751910552bf'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:43.619784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7a382b92-bdd9-4aed-b607-b751910552bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '076249e805944633bc76d15cc9eaff2b', 'request_id': 'req-7a382b92-bdd9-4aed-b607-b751910552bf'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:43.649727 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7a382b92-bdd9-4aed-b607-b751910552bf admin admin] The token cache is being invalidate because project 076249e805944633bc76d15cc9eaff2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:43.650225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7a382b92-bdd9-4aed-b607-b751910552bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:43.651005 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7a382b92-bdd9-4aed-b607-b751910552bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:43.653112 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 920/1842] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:43 2022] DELETE /identity/v3/projects/076249e805944633bc76d15cc9eaff2b => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:26:43.654986 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96193e6-f5e8-435e-b5e8-bfe9c3ed4281 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:43.655260 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96193e6-f5e8-435e-b5e8-bfe9c3ed4281 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:43.655551 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96193e6-f5e8-435e-b5e8-bfe9c3ed4281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:43.707081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-f96193e6-f5e8-435e-b5e8-bfe9c3ed4281 None None] MFA Rules not processed for user `c1361321974f4af080f4618a5ed1c8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:43.708583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f96193e6-f5e8-435e-b5e8-bfe9c3ed4281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:43.719605 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73e9cc-6adc-441c-b079-cfc79551bfcb admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:43.735950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a069a65-6120-4a38-8d34-59e2f4df88fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:43.736718 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a069a65-6120-4a38-8d34-59e2f4df88fc admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:43.737641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a069a65-6120-4a38-8d34-59e2f4df88fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:43.738018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a069a65-6120-4a38-8d34-59e2f4df88fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:43.738768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a069a65-6120-4a38-8d34-59e2f4df88fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:43.740568 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a069a65-6120-4a38-8d34-59e2f4df88fc admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:43.776714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 923/1843] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:26:43.785360 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b83c5-49f3-41f7-8caa-7b99f5566ed4 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:43.785874 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b83c5-49f3-41f7-8caa-7b99f5566ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:43.786392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b83c5-49f3-41f7-8caa-7b99f5566ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:43.847934 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-a89b83c5-49f3-41f7-8caa-7b99f5566ed4 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:43.853253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-a89b83c5-49f3-41f7-8caa-7b99f5566ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:43.876041 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 924/1844] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:43.877072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b83c5-49f3-41f7-8caa-7b99f5566ed4 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:43.879906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-a89b83c5-49f3-41f7-8caa-7b99f5566ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:43.898017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 921/1845] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:43 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:43.898017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42126f59-de7d-4c59-920e-5d39e3688206 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:43.898848 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42126f59-de7d-4c59-920e-5d39e3688206 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:43.898848 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42126f59-de7d-4c59-920e-5d39e3688206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:43.900683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b83c5-49f3-41f7-8caa-7b99f5566ed4 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:43.916865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9eceb-b98e-4c31-b0c5-daf84f8717f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:43.917887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9eceb-b98e-4c31-b0c5-daf84f8717f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:43.918263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9eceb-b98e-4c31-b0c5-daf84f8717f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:43.918696 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9eceb-b98e-4c31-b0c5-daf84f8717f4 admin admin] PATH_INFO: `/v3/projects/ce4d3f3bce174b3e924d517763b02a9a/users/4e8fb30a505144ad8685cca92446269a/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:43.923808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9eceb-b98e-4c31-b0c5-daf84f8717f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce4d3f3bce174b3e924d517763b02a9a, user_id=4e8fb30a505144ad8685cca92446269a, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:43.927150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9eceb-b98e-4c31-b0c5-daf84f8717f4 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:43.942321 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 925/1846] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:43 2022] PUT /identity/v3/projects/ce4d3f3bce174b3e924d517763b02a9a/users/4e8fb30a505144ad8685cca92446269a/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:43.943878 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9eceb-b98e-4c31-b0c5-daf84f8717f4 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:43.945490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-87c9eceb-b98e-4c31-b0c5-daf84f8717f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:43.947811 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-42126f59-de7d-4c59-920e-5d39e3688206 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:43.948760 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-42126f59-de7d-4c59-920e-5d39e3688206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:44.063032 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 922/1847] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:44.064732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308692ae-b1cc-4e1c-b8f7-d50a434b180a None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:44.065020 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308692ae-b1cc-4e1c-b8f7-d50a434b180a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:44.065297 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308692ae-b1cc-4e1c-b8f7-d50a434b180a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:44.098224 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-308692ae-b1cc-4e1c-b8f7-d50a434b180a None None] MFA Rules not processed for user `4e8fb30a505144ad8685cca92446269a`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:44.099804 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-308692ae-b1cc-4e1c-b8f7-d50a434b180a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:44.107862 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9eceb-b98e-4c31-b0c5-daf84f8717f4 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:44.111019 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:44.117979 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb36f3a-c505-4d2f-9bfa-1cce338c1687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:44.119187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb36f3a-c505-4d2f-9bfa-1cce338c1687 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:44.120019 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb36f3a-c505-4d2f-9bfa-1cce338c1687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:44.120283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb36f3a-c505-4d2f-9bfa-1cce338c1687 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:44.121963 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb36f3a-c505-4d2f-9bfa-1cce338c1687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:44.126396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb36f3a-c505-4d2f-9bfa-1cce338c1687 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:44.195596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 926/1848] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:43 2022] POST /identity/v3/projects => generated 403 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:44.196374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb36f3a-c505-4d2f-9bfa-1cce338c1687 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:44.199676 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 923/1849] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:44.200723 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308692ae-b1cc-4e1c-b8f7-d50a434b180a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:44.218613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308692ae-b1cc-4e1c-b8f7-d50a434b180a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:44.241435 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:44.296205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb36f3a-c505-4d2f-9bfa-1cce338c1687 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:44.304494 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471b45e-dc27-4415-9b5a-a10d82b1c9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:44.305758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471b45e-dc27-4415-9b5a-a10d82b1c9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:44.306040 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471b45e-dc27-4415-9b5a-a10d82b1c9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:44.306310 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471b45e-dc27-4415-9b5a-a10d82b1c9f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:44.306994 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471b45e-dc27-4415-9b5a-a10d82b1c9f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:26:44.307348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471b45e-dc27-4415-9b5a-a10d82b1c9f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:44.309272 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4471b45e-dc27-4415-9b5a-a10d82b1c9f0 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:44.311799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964a9d1-2f25-4e7a-a184-7624d45eda0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:44.312622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964a9d1-2f25-4e7a-a184-7624d45eda0c admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:44.313003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964a9d1-2f25-4e7a-a184-7624d45eda0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:44.313377 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964a9d1-2f25-4e7a-a184-7624d45eda0c admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:44.314384 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964a9d1-2f25-4e7a-a184-7624d45eda0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:44.317342 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964a9d1-2f25-4e7a-a184-7624d45eda0c admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:44.328270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 927/1850] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:26:44.329406 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471b45e-dc27-4415-9b5a-a10d82b1c9f0 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:44.333501 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4471b45e-dc27-4415-9b5a-a10d82b1c9f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:44.351319 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 924/1851] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:44 2022] POST /identity/v3/users => generated 308 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:44.351319 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964a9d1-2f25-4e7a-a184-7624d45eda0c admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:44.363463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964a9d1-2f25-4e7a-a184-7624d45eda0c admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:44.380854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7548585-2499-45a7-9b79-ba15f18641da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:44.381647 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7548585-2499-45a7-9b79-ba15f18641da admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:44.381979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7548585-2499-45a7-9b79-ba15f18641da admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:44.382297 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7548585-2499-45a7-9b79-ba15f18641da admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:44.383025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7548585-2499-45a7-9b79-ba15f18641da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:44.385273 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7548585-2499-45a7-9b79-ba15f18641da admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:44.404901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471b45e-dc27-4415-9b5a-a10d82b1c9f0 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:44.409104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0599a54-c38e-4973-beba-2c717f42a6dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:44.409890 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0599a54-c38e-4973-beba-2c717f42a6dd service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:44.410235 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0599a54-c38e-4973-beba-2c717f42a6dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:44.410574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0599a54-c38e-4973-beba-2c717f42a6dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:44.450761 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 925/1852] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:44 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:44.451638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7548585-2499-45a7-9b79-ba15f18641da admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:44.491641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0599a54-c38e-4973-beba-2c717f42a6dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:44.493166 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0599a54-c38e-4973-beba-2c717f42a6dd service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:44.509205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 928/1853] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:44.510133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0599a54-c38e-4973-beba-2c717f42a6dd service cinder] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:44.523335 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0599a54-c38e-4973-beba-2c717f42a6dd service cinder] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:44.575632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7548585-2499-45a7-9b79-ba15f18641da admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:44.585737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e068a-0ac5-4950-806d-4fb55933a62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:44.587628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e068a-0ac5-4950-806d-4fb55933a62c admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:44.588163 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e068a-0ac5-4950-806d-4fb55933a62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:44.588664 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e068a-0ac5-4950-806d-4fb55933a62c admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:44.589570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e068a-0ac5-4950-806d-4fb55933a62c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:44.591939 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e068a-0ac5-4950-806d-4fb55933a62c admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:44.608185 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 926/1854] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:44.612981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e068a-0ac5-4950-806d-4fb55933a62c admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:44.643818 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e068a-0ac5-4950-806d-4fb55933a62c admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:44.644528 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e288ae-38a5-4ad9-b67a-af46ddcba022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:44.645134 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e288ae-38a5-4ad9-b67a-af46ddcba022 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:44.645413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e288ae-38a5-4ad9-b67a-af46ddcba022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:44.645828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e288ae-38a5-4ad9-b67a-af46ddcba022 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:44.646470 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e288ae-38a5-4ad9-b67a-af46ddcba022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:44.648461 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e288ae-38a5-4ad9-b67a-af46ddcba022 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:44.666138 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f164b50-610c-4ada-bf8f-1dd391b49bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:44.666847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f164b50-610c-4ada-bf8f-1dd391b49bf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:44.667132 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f164b50-610c-4ada-bf8f-1dd391b49bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:44.667416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f164b50-610c-4ada-bf8f-1dd391b49bf1 admin admin] PATH_INFO: `/v3/projects/aea447b894424a7394af909239b2e21a/users/f562db3cb3924963aa582c6e3e76adfc/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:44.671693 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:44.682000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:44.692649 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 929/1855] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:44 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:44.698035 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e288ae-38a5-4ad9-b67a-af46ddcba022 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:44.706010 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f164b50-610c-4ada-bf8f-1dd391b49bf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea447b894424a7394af909239b2e21a, user_id=f562db3cb3924963aa582c6e3e76adfc, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:44.708999 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f164b50-610c-4ada-bf8f-1dd391b49bf1 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:44.731072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 927/1856] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:44 2022] PUT /identity/v3/projects/aea447b894424a7394af909239b2e21a/users/f562db3cb3924963aa582c6e3e76adfc/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:44.734240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e288ae-38a5-4ad9-b67a-af46ddcba022 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:44.735262 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f164b50-610c-4ada-bf8f-1dd391b49bf1 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:44.845695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f164b50-610c-4ada-bf8f-1dd391b49bf1 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:44.853420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73ee37-d1e0-4b75-9632-fa3136bbb3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:44.854198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73ee37-d1e0-4b75-9632-fa3136bbb3ef admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:44.854575 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73ee37-d1e0-4b75-9632-fa3136bbb3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:44.854979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d73ee37-d1e0-4b75-9632-fa3136bbb3ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:44.855948 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73ee37-d1e0-4b75-9632-fa3136bbb3ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:44.858099 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d73ee37-d1e0-4b75-9632-fa3136bbb3ef admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:44.873002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 928/1857] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:26:44.877818 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73ee37-d1e0-4b75-9632-fa3136bbb3ef admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:44.920634 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d73ee37-d1e0-4b75-9632-fa3136bbb3ef admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:44.927538 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e43cb3-2853-4059-b1b4-f6f36a1e7dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:44.929552 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43cb3-2853-4059-b1b4-f6f36a1e7dbc admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:44.930261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43cb3-2853-4059-b1b4-f6f36a1e7dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:44.931460 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43cb3-2853-4059-b1b4-f6f36a1e7dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:44.932843 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e43cb3-2853-4059-b1b4-f6f36a1e7dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:44.937434 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e43cb3-2853-4059-b1b4-f6f36a1e7dbc admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:44.939235 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9388d68-f0d2-4fc2-9523-d4c5ac9bbe5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:44.941011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9388d68-f0d2-4fc2-9523-d4c5ac9bbe5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:44.941314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9388d68-f0d2-4fc2-9523-d4c5ac9bbe5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:44.941592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9388d68-f0d2-4fc2-9523-d4c5ac9bbe5d admin admin] PATH_INFO: `/v3/projects/aea447b894424a7394af909239b2e21a/users/f562db3cb3924963aa582c6e3e76adfc/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:44.944988 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9388d68-f0d2-4fc2-9523-d4c5ac9bbe5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea447b894424a7394af909239b2e21a, user_id=f562db3cb3924963aa582c6e3e76adfc, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:44.947833 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9388d68-f0d2-4fc2-9523-d4c5ac9bbe5d admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:44.976472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 930/1858] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:44.977376 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e43cb3-2853-4059-b1b4-f6f36a1e7dbc admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:44.978970 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 929/1859] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:44 2022] PUT /identity/v3/projects/aea447b894424a7394af909239b2e21a/users/f562db3cb3924963aa582c6e3e76adfc/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:26:45.022475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e43cb3-2853-4059-b1b4-f6f36a1e7dbc admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:45.152413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bfec9-16eb-4ad7-b54e-e10677b2d166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:45.155671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bfec9-16eb-4ad7-b54e-e10677b2d166 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:45.155830 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bfec9-16eb-4ad7-b54e-e10677b2d166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:45.157983 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bfec9-16eb-4ad7-b54e-e10677b2d166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:45.157983 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29bfec9-16eb-4ad7-b54e-e10677b2d166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:45.158770 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29bfec9-16eb-4ad7-b54e-e10677b2d166 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:45.201389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 931/1860] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:45.206320 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9388d68-f0d2-4fc2-9523-d4c5ac9bbe5d admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:45.221334 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9388d68-f0d2-4fc2-9523-d4c5ac9bbe5d admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:45.237524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03802c2b-de39-44e5-8c43-da54fd616bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:45.237524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03802c2b-de39-44e5-8c43-da54fd616bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:45.237524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03802c2b-de39-44e5-8c43-da54fd616bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:45.237524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03802c2b-de39-44e5-8c43-da54fd616bd3 admin admin] PATH_INFO: `/v3/projects/7c663e9a8ec74db2aa769769fcb04118/users/a4a1ef092eeb493886ad12f3bedca097/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:45.273964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03802c2b-de39-44e5-8c43-da54fd616bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c663e9a8ec74db2aa769769fcb04118, user_id=a4a1ef092eeb493886ad12f3bedca097, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:45.277612 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03802c2b-de39-44e5-8c43-da54fd616bd3 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:45.322816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 930/1861] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:45 2022] PUT /identity/v3/projects/7c663e9a8ec74db2aa769769fcb04118/users/a4a1ef092eeb493886ad12f3bedca097/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:45.341006 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cbfb7-91fe-4aec-ae13-15d8be468e36 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:45.341315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cbfb7-91fe-4aec-ae13-15d8be468e36 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:45.341601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3cbfb7-91fe-4aec-ae13-15d8be468e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:45.393475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-7d3cbfb7-91fe-4aec-ae13-15d8be468e36 None None] MFA Rules not processed for user `a4a1ef092eeb493886ad12f3bedca097`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:45.394901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-7d3cbfb7-91fe-4aec-ae13-15d8be468e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:45.552673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 932/1862] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:26:45.567868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03802c2b-de39-44e5-8c43-da54fd616bd3 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:45.570106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-03802c2b-de39-44e5-8c43-da54fd616bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:45.652860 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03802c2b-de39-44e5-8c43-da54fd616bd3 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:45.657252 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4718393-775e-407a-ae7c-59c91075a7fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:45.658092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4718393-775e-407a-ae7c-59c91075a7fb service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:45.658375 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4718393-775e-407a-ae7c-59c91075a7fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:45.658707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4718393-775e-407a-ae7c-59c91075a7fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:45.667888 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4718393-775e-407a-ae7c-59c91075a7fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:45.669363 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4718393-775e-407a-ae7c-59c91075a7fb service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:45.680435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 931/1863] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:45.691763 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cbfb7-91fe-4aec-ae13-15d8be468e36 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:45.723609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3cbfb7-91fe-4aec-ae13-15d8be468e36 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:45.736866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0699f-875e-457c-a211-8df5fe26ad55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:45.737703 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0699f-875e-457c-a211-8df5fe26ad55 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:45.738001 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0699f-875e-457c-a211-8df5fe26ad55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:45.738316 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0699f-875e-457c-a211-8df5fe26ad55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:45.741803 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-7dd0699f-875e-457c-a211-8df5fe26ad55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:45.776388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:45.861975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0699f-875e-457c-a211-8df5fe26ad55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:45.863713 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0699f-875e-457c-a211-8df5fe26ad55 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:45.874788 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 933/1864] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:46.522996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dd3cb-2007-477b-95eb-0a4a3ae832c1 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:46.524108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dd3cb-2007-477b-95eb-0a4a3ae832c1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:46.524554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069dd3cb-2007-477b-95eb-0a4a3ae832c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:46.562464 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-069dd3cb-2007-477b-95eb-0a4a3ae832c1 None None] MFA Rules not processed for user `f562db3cb3924963aa582c6e3e76adfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:46.563782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-069dd3cb-2007-477b-95eb-0a4a3ae832c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:46.627461 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:46.682725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 932/1865] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:46 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:47.758530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748458e-d152-48b1-b2b9-947dff8cd2e7 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:47.758530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748458e-d152-48b1-b2b9-947dff8cd2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:47.759044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748458e-d152-48b1-b2b9-947dff8cd2e7 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:47.761581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 934/1866] 10.222.0.17 () {54 vars in 921 bytes} [Sun Aug 7 19:26:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:26:47.782502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8949fac-2612-4534-900a-7247ac8756a0 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:47.783065 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8949fac-2612-4534-900a-7247ac8756a0 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:47.783484 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8949fac-2612-4534-900a-7247ac8756a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:47.813543 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-e8949fac-2612-4534-900a-7247ac8756a0 None None] MFA Rules not processed for user `143c30b4201645279564e72ab3288adc`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:47.815368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e8949fac-2612-4534-900a-7247ac8756a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:47.908409 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 933/1867] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Aug 7 19:26:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:47.958284 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748458e-d152-48b1-b2b9-947dff8cd2e7 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:47.960690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4748458e-d152-48b1-b2b9-947dff8cd2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:47.993939 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748458e-d152-48b1-b2b9-947dff8cd2e7 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:48.002120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599e614-12fc-4bee-bcf9-923f3c135c01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '143c30b4201645279564e72ab3288adc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:48.003028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599e614-12fc-4bee-bcf9-923f3c135c01 service placement] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:48.003577 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599e614-12fc-4bee-bcf9-923f3c135c01 service placement] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:48.003854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599e614-12fc-4bee-bcf9-923f3c135c01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:48.014332 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599e614-12fc-4bee-bcf9-923f3c135c01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:48.017079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599e614-12fc-4bee-bcf9-923f3c135c01 service placement] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:48.040652 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 935/1868] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:48.661690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47871e4a-25a1-4949-b6f6-c429b812839a None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:48.661971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47871e4a-25a1-4949-b6f6-c429b812839a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:48.662249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47871e4a-25a1-4949-b6f6-c429b812839a None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:48.664547 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 934/1869] 10.222.0.17 () {54 vars in 921 bytes} [Sun Aug 7 19:26:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:26:48.674022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea16d8-9268-4e46-970e-c9ce7e4eedc9 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:48.674283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea16d8-9268-4e46-970e-c9ce7e4eedc9 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:48.675018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ea16d8-9268-4e46-970e-c9ce7e4eedc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:48.706196 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-66ea16d8-9268-4e46-970e-c9ce7e4eedc9 None None] MFA Rules not processed for user `143c30b4201645279564e72ab3288adc`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:48.707797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-66ea16d8-9268-4e46-970e-c9ce7e4eedc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:48.718810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 936/1870] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Aug 7 19:26:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:48.731974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47871e4a-25a1-4949-b6f6-c429b812839a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:48.749627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47871e4a-25a1-4949-b6f6-c429b812839a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:48.750635 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea16d8-9268-4e46-970e-c9ce7e4eedc9 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:48.755864 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a922e6-4479-43b6-97e8-6988a22c44d7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '143c30b4201645279564e72ab3288adc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:48.756788 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a922e6-4479-43b6-97e8-6988a22c44d7 service placement] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:48.757137 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a922e6-4479-43b6-97e8-6988a22c44d7 service placement] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:48.757509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a922e6-4479-43b6-97e8-6988a22c44d7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:48.768460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a922e6-4479-43b6-97e8-6988a22c44d7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:48.770346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a922e6-4479-43b6-97e8-6988a22c44d7 service placement] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:48.782085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 935/1871] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:26:48.837017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ea16d8-9268-4e46-970e-c9ce7e4eedc9 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:48.842226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7858bf0-8174-4548-aa85-008d14c6467e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:48.842878 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7858bf0-8174-4548-aa85-008d14c6467e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:48.843151 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7858bf0-8174-4548-aa85-008d14c6467e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:48.843591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7858bf0-8174-4548-aa85-008d14c6467e admin admin] PATH_INFO: `/v3/users/c1361321974f4af080f4618a5ed1c8a7` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:48.844802 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7858bf0-8174-4548-aa85-008d14c6467e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1361321974f4af080f4618a5ed1c8a7)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:48.846776 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7858bf0-8174-4548-aa85-008d14c6467e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:48.897878 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a7858bf0-8174-4548-aa85-008d14c6467e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1361321974f4af080f4618a5ed1c8a7'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:48.910898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a7858bf0-8174-4548-aa85-008d14c6467e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1361321974f4af080f4618a5ed1c8a7'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:48.918266 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a7858bf0-8174-4548-aa85-008d14c6467e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1361321974f4af080f4618a5ed1c8a7'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:48.919055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a7858bf0-8174-4548-aa85-008d14c6467e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1361321974f4af080f4618a5ed1c8a7'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:48.919735 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a7858bf0-8174-4548-aa85-008d14c6467e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1361321974f4af080f4618a5ed1c8a7'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:48.936231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 937/1872] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:48 2022] DELETE /identity/v3/users/c1361321974f4af080f4618a5ed1c8a7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:26:49.133241 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a922e6-4479-43b6-97e8-6988a22c44d7 service placement] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:49.146004 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a922e6-4479-43b6-97e8-6988a22c44d7 service placement] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:49.230832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a0064-dfc0-466c-995f-33040a54595c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:49.231487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a0064-dfc0-466c-995f-33040a54595c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:49.231765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a0064-dfc0-466c-995f-33040a54595c admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:49.232080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a0064-dfc0-466c-995f-33040a54595c admin admin] PATH_INFO: `/v3/projects/66c4e94b6b354f60984ec37dff8b6af5` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:49.233227 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a0064-dfc0-466c-995f-33040a54595c admin admin] RBAC: Authorizing `identity:delete_project(project_id=66c4e94b6b354f60984ec37dff8b6af5)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:49.235166 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a0064-dfc0-466c-995f-33040a54595c admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:49.298964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d39a0064-dfc0-466c-995f-33040a54595c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66c4e94b6b354f60984ec37dff8b6af5', 'request_id': 'req-d39a0064-dfc0-466c-995f-33040a54595c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:49.307020 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:49.311488 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d39a0064-dfc0-466c-995f-33040a54595c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66c4e94b6b354f60984ec37dff8b6af5', 'request_id': 'req-d39a0064-dfc0-466c-995f-33040a54595c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:49.320630 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d39a0064-dfc0-466c-995f-33040a54595c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66c4e94b6b354f60984ec37dff8b6af5', 'request_id': 'req-d39a0064-dfc0-466c-995f-33040a54595c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:49.340017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d39a0064-dfc0-466c-995f-33040a54595c admin admin] The token cache is being invalidate because project 66c4e94b6b354f60984ec37dff8b6af5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:49.340553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d39a0064-dfc0-466c-995f-33040a54595c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:49.341442 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d39a0064-dfc0-466c-995f-33040a54595c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:49.343931 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 936/1873] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:49 2022] DELETE /identity/v3/projects/66c4e94b6b354f60984ec37dff8b6af5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:49.347910 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7858bf0-8174-4548-aa85-008d14c6467e admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:49.349823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-a7858bf0-8174-4548-aa85-008d14c6467e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:49.365289 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7858bf0-8174-4548-aa85-008d14c6467e admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:49.381699 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:49.454598 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef9fca-59f9-4cdb-b5e5-09adcaa88fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:49.455256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef9fca-59f9-4cdb-b5e5-09adcaa88fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:49.455541 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef9fca-59f9-4cdb-b5e5-09adcaa88fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:49.455812 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef9fca-59f9-4cdb-b5e5-09adcaa88fe8 admin admin] PATH_INFO: `/v3/users/f562db3cb3924963aa582c6e3e76adfc` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:49.457084 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef9fca-59f9-4cdb-b5e5-09adcaa88fe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f562db3cb3924963aa582c6e3e76adfc)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:49.459032 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef9fca-59f9-4cdb-b5e5-09adcaa88fe8 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:49.518793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-21ef9fca-59f9-4cdb-b5e5-09adcaa88fe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f562db3cb3924963aa582c6e3e76adfc'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:49.531799 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-21ef9fca-59f9-4cdb-b5e5-09adcaa88fe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f562db3cb3924963aa582c6e3e76adfc'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:49.539356 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-21ef9fca-59f9-4cdb-b5e5-09adcaa88fe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f562db3cb3924963aa582c6e3e76adfc'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:49.540121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-21ef9fca-59f9-4cdb-b5e5-09adcaa88fe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f562db3cb3924963aa582c6e3e76adfc'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:49.540723 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-21ef9fca-59f9-4cdb-b5e5-09adcaa88fe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f562db3cb3924963aa582c6e3e76adfc'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:49.557504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 938/1874] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:49 2022] DELETE /identity/v3/users/f562db3cb3924963aa582c6e3e76adfc => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:26:49.752238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a0064-dfc0-466c-995f-33040a54595c admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:49.754020 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-d39a0064-dfc0-466c-995f-33040a54595c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:49.777666 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a0064-dfc0-466c-995f-33040a54595c admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:49.843159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f3c23-35de-42eb-aa6e-9e123bbf5334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:49.843970 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f3c23-35de-42eb-aa6e-9e123bbf5334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:49.844366 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f3c23-35de-42eb-aa6e-9e123bbf5334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:49.844641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f3c23-35de-42eb-aa6e-9e123bbf5334 admin admin] PATH_INFO: `/v3/projects/aea447b894424a7394af909239b2e21a` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:49.845875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f3c23-35de-42eb-aa6e-9e123bbf5334 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aea447b894424a7394af909239b2e21a)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:49.848124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f3c23-35de-42eb-aa6e-9e123bbf5334 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:49.903817 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-687f3c23-35de-42eb-aa6e-9e123bbf5334 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aea447b894424a7394af909239b2e21a', 'request_id': 'req-687f3c23-35de-42eb-aa6e-9e123bbf5334'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:49.910009 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-687f3c23-35de-42eb-aa6e-9e123bbf5334 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aea447b894424a7394af909239b2e21a', 'request_id': 'req-687f3c23-35de-42eb-aa6e-9e123bbf5334'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:49.916928 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-687f3c23-35de-42eb-aa6e-9e123bbf5334 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aea447b894424a7394af909239b2e21a', 'request_id': 'req-687f3c23-35de-42eb-aa6e-9e123bbf5334'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:49.932032 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-687f3c23-35de-42eb-aa6e-9e123bbf5334 admin admin] The token cache is being invalidate because project aea447b894424a7394af909239b2e21a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:49.932401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-687f3c23-35de-42eb-aa6e-9e123bbf5334 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:49.933085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-687f3c23-35de-42eb-aa6e-9e123bbf5334 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:49.934670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 937/1875] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:49 2022] DELETE /identity/v3/projects/aea447b894424a7394af909239b2e21a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:49.940014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5226c7-65d5-435b-8543-d02099b31972 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:49.940535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5226c7-65d5-435b-8543-d02099b31972 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:49.940634 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5226c7-65d5-435b-8543-d02099b31972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:49.960305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-7e5226c7-65d5-435b-8543-d02099b31972 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:49.961522 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-7e5226c7-65d5-435b-8543-d02099b31972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:50.021989 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 939/1876] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:50.030084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b676b3-bec8-4e19-9a41-2cafdc7cf0c1 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:50.030375 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b676b3-bec8-4e19-9a41-2cafdc7cf0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:50.030651 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b676b3-bec8-4e19-9a41-2cafdc7cf0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:50.051962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-f9b676b3-bec8-4e19-9a41-2cafdc7cf0c1 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:50.053211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f9b676b3-bec8-4e19-9a41-2cafdc7cf0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:50.062189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 938/1877] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:50.067021 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5226c7-65d5-435b-8543-d02099b31972 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:50.077409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5226c7-65d5-435b-8543-d02099b31972 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:50.082576 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94ec5e-2cd9-4235-9a31-e2a2b8ee8cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:50.083194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94ec5e-2cd9-4235-9a31-e2a2b8ee8cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:50.083469 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94ec5e-2cd9-4235-9a31-e2a2b8ee8cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:50.083739 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e94ec5e-2cd9-4235-9a31-e2a2b8ee8cb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:50.084430 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94ec5e-2cd9-4235-9a31-e2a2b8ee8cb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:26:50.084732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94ec5e-2cd9-4235-9a31-e2a2b8ee8cb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:50.086318 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e94ec5e-2cd9-4235-9a31-e2a2b8ee8cb7 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:50.099218 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 940/1878] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:26:50.103830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b676b3-bec8-4e19-9a41-2cafdc7cf0c1 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:50.113623 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b676b3-bec8-4e19-9a41-2cafdc7cf0c1 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:50.118734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8532de-52df-4818-97ff-eadc7d812257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:50.119356 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8532de-52df-4818-97ff-eadc7d812257 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:50.119634 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8532de-52df-4818-97ff-eadc7d812257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:50.119913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8532de-52df-4818-97ff-eadc7d812257 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:50.120582 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8532de-52df-4818-97ff-eadc7d812257 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:50.122525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8532de-52df-4818-97ff-eadc7d812257 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:50.152907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 939/1879] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:50 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:50.157379 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94ec5e-2cd9-4235-9a31-e2a2b8ee8cb7 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:50.168038 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e94ec5e-2cd9-4235-9a31-e2a2b8ee8cb7 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:50.190712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:50.240492 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50047e8-6504-46aa-af27-2d14c917c0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:50.241122 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50047e8-6504-46aa-af27-2d14c917c0c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:50.241448 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50047e8-6504-46aa-af27-2d14c917c0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:50.241792 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50047e8-6504-46aa-af27-2d14c917c0c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:50.242473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50047e8-6504-46aa-af27-2d14c917c0c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:50.244473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50047e8-6504-46aa-af27-2d14c917c0c4 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:50.270157 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 941/1880] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:50 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:50.272726 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8532de-52df-4818-97ff-eadc7d812257 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:50.284392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8532de-52df-4818-97ff-eadc7d812257 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:50.290100 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304eea4-7df3-4f0d-9042-cba6d160710e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:50.290810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304eea4-7df3-4f0d-9042-cba6d160710e admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:50.291098 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304eea4-7df3-4f0d-9042-cba6d160710e admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:50.291366 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304eea4-7df3-4f0d-9042-cba6d160710e admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:50.292159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304eea4-7df3-4f0d-9042-cba6d160710e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:50.293832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e304eea4-7df3-4f0d-9042-cba6d160710e admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:50.306127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 940/1881] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:50.310543 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50047e8-6504-46aa-af27-2d14c917c0c4 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:50.322365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50047e8-6504-46aa-af27-2d14c917c0c4 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:50.337421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2289da8-959e-48e5-aa8f-636287151103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:50.338070 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2289da8-959e-48e5-aa8f-636287151103 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:50.338376 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2289da8-959e-48e5-aa8f-636287151103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:50.338684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2289da8-959e-48e5-aa8f-636287151103 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:50.339346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2289da8-959e-48e5-aa8f-636287151103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:50.341000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2289da8-959e-48e5-aa8f-636287151103 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:50.352997 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 942/1882] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:50.357316 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304eea4-7df3-4f0d-9042-cba6d160710e admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:50.368344 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304eea4-7df3-4f0d-9042-cba6d160710e admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:50.373945 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4248dd8-63c3-4e23-8e6c-78154b758627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:50.374833 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4248dd8-63c3-4e23-8e6c-78154b758627 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:50.375258 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4248dd8-63c3-4e23-8e6c-78154b758627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:50.375670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4248dd8-63c3-4e23-8e6c-78154b758627 admin admin] PATH_INFO: `/v3/projects/4716c702a68f44939c98995380cb83c2/users/f8e837d350b04a87880cb9975ecd177d/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:50.393413 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4248dd8-63c3-4e23-8e6c-78154b758627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4716c702a68f44939c98995380cb83c2, user_id=f8e837d350b04a87880cb9975ecd177d, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:50.396667 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4248dd8-63c3-4e23-8e6c-78154b758627 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:50.422908 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 941/1883] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:50 2022] PUT /identity/v3/projects/4716c702a68f44939c98995380cb83c2/users/f8e837d350b04a87880cb9975ecd177d/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:26:50.427456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea98d9-b386-4f29-89b5-e718aa5b5dfb None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:50.427888 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea98d9-b386-4f29-89b5-e718aa5b5dfb None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:50.428309 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea98d9-b386-4f29-89b5-e718aa5b5dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:50.492877 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-96ea98d9-b386-4f29-89b5-e718aa5b5dfb None None] MFA Rules not processed for user `f8e837d350b04a87880cb9975ecd177d`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:50.495289 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-96ea98d9-b386-4f29-89b5-e718aa5b5dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:50.598573 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 943/1884] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:50.603429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4248dd8-63c3-4e23-8e6c-78154b758627 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:50.623757 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:50.747688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4248dd8-63c3-4e23-8e6c-78154b758627 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:50.752820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab43b2b-7475-4624-bc23-dc9cd3d7741b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:50.753967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab43b2b-7475-4624-bc23-dc9cd3d7741b admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:50.753967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab43b2b-7475-4624-bc23-dc9cd3d7741b admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:50.753967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab43b2b-7475-4624-bc23-dc9cd3d7741b admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:50.754630 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab43b2b-7475-4624-bc23-dc9cd3d7741b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:50.756608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab43b2b-7475-4624-bc23-dc9cd3d7741b admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:50.791227 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 942/1885] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:50 2022] POST /identity/v3/projects => generated 389 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:50.795526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea98d9-b386-4f29-89b5-e718aa5b5dfb None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:50.807620 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea98d9-b386-4f29-89b5-e718aa5b5dfb None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:50.867079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670c203-8c28-47d8-8ecb-23be7f3844af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:50.867606 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670c203-8c28-47d8-8ecb-23be7f3844af admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:50.867882 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670c203-8c28-47d8-8ecb-23be7f3844af admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:50.868178 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b670c203-8c28-47d8-8ecb-23be7f3844af admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:50.868825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670c203-8c28-47d8-8ecb-23be7f3844af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:50.870742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b670c203-8c28-47d8-8ecb-23be7f3844af admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:50.893386 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 944/1886] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:50 2022] POST /identity/v3/users => generated 301 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:26:50.897371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab43b2b-7475-4624-bc23-dc9cd3d7741b admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:50.913846 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab43b2b-7475-4624-bc23-dc9cd3d7741b admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:50.934392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67079873-cd32-4f36-ae6c-22c692aa98c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:50.935127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67079873-cd32-4f36-ae6c-22c692aa98c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:50.935414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67079873-cd32-4f36-ae6c-22c692aa98c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:50.935763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67079873-cd32-4f36-ae6c-22c692aa98c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:50.936512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67079873-cd32-4f36-ae6c-22c692aa98c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:50.938336 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67079873-cd32-4f36-ae6c-22c692aa98c6 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:50.951570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 943/1887] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:50.956170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670c203-8c28-47d8-8ecb-23be7f3844af admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:50.967125 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b670c203-8c28-47d8-8ecb-23be7f3844af admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:50.973207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e03429-7d3c-477c-89a6-f9489814deab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:50.974074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e03429-7d3c-477c-89a6-f9489814deab admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:50.974314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e03429-7d3c-477c-89a6-f9489814deab admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:50.974596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e03429-7d3c-477c-89a6-f9489814deab admin admin] PATH_INFO: `/v3/projects/c8c5aa765c7a414d89e4deff5d807fdb/users/49715d7f74724d3fbbf7fc9ab8624c5b/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:50.991889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e03429-7d3c-477c-89a6-f9489814deab admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c5aa765c7a414d89e4deff5d807fdb, user_id=49715d7f74724d3fbbf7fc9ab8624c5b, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:50.994855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e03429-7d3c-477c-89a6-f9489814deab admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:51.004683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:51.010748 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 945/1888] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:50 2022] PUT /identity/v3/projects/c8c5aa765c7a414d89e4deff5d807fdb/users/49715d7f74724d3fbbf7fc9ab8624c5b/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:51.016521 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67079873-cd32-4f36-ae6c-22c692aa98c6 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:51.030893 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67079873-cd32-4f36-ae6c-22c692aa98c6 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:51.097127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dae752-00cd-4326-974f-9e4e753e80e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:51.097771 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dae752-00cd-4326-974f-9e4e753e80e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:51.098053 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dae752-00cd-4326-974f-9e4e753e80e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:51.098331 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dae752-00cd-4326-974f-9e4e753e80e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:51.098985 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dae752-00cd-4326-974f-9e4e753e80e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:51.100652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dae752-00cd-4326-974f-9e4e753e80e7 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:51.111644 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 944/1889] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:51.116432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e03429-7d3c-477c-89a6-f9489814deab admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:51.127684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e03429-7d3c-477c-89a6-f9489814deab admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:51.133308 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c76eb-3bf3-47b9-bb6f-c58284975d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:51.134124 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c76eb-3bf3-47b9-bb6f-c58284975d79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:51.134402 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c76eb-3bf3-47b9-bb6f-c58284975d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:51.134676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c76eb-3bf3-47b9-bb6f-c58284975d79 admin admin] PATH_INFO: `/v3/projects/c8c5aa765c7a414d89e4deff5d807fdb/users/49715d7f74724d3fbbf7fc9ab8624c5b/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:51.136881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c76eb-3bf3-47b9-bb6f-c58284975d79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8c5aa765c7a414d89e4deff5d807fdb, user_id=49715d7f74724d3fbbf7fc9ab8624c5b, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:51.139472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c76eb-3bf3-47b9-bb6f-c58284975d79 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:51.152161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 946/1890] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:51 2022] PUT /identity/v3/projects/c8c5aa765c7a414d89e4deff5d807fdb/users/49715d7f74724d3fbbf7fc9ab8624c5b/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:26:51.165021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb8af2-7774-4c22-ab76-cd0e991b3a75 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:51.165545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb8af2-7774-4c22-ab76-cd0e991b3a75 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:51.165545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb8af2-7774-4c22-ab76-cd0e991b3a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:51.202889 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-03eb8af2-7774-4c22-ab76-cd0e991b3a75 None None] MFA Rules not processed for user `49715d7f74724d3fbbf7fc9ab8624c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:51.204307 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-03eb8af2-7774-4c22-ab76-cd0e991b3a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:51.288431 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 945/1891] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:51 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:51.297180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c76eb-3bf3-47b9-bb6f-c58284975d79 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:51.299588 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-fa0c76eb-3bf3-47b9-bb6f-c58284975d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:51.375030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c76eb-3bf3-47b9-bb6f-c58284975d79 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:51.378950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b7864-0f7e-4734-bedd-2d3102d6fd80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:51.379577 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b7864-0f7e-4734-bedd-2d3102d6fd80 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:51.379854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b7864-0f7e-4734-bedd-2d3102d6fd80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:51.380147 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b7864-0f7e-4734-bedd-2d3102d6fd80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:51.389691 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b7864-0f7e-4734-bedd-2d3102d6fd80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:51.391164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b7864-0f7e-4734-bedd-2d3102d6fd80 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:51.401171 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 947/1892] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:51 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:51.544816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb8af2-7774-4c22-ab76-cd0e991b3a75 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:51.564837 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:51.763569 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb8af2-7774-4c22-ab76-cd0e991b3a75 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:51.777139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b474-1418-4e37-b140-6c51eea73f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:51.777865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b474-1418-4e37-b140-6c51eea73f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:51.778139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b474-1418-4e37-b140-6c51eea73f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:51.778503 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b474-1418-4e37-b140-6c51eea73f71 admin admin] PATH_INFO: `/v3/users/f8e837d350b04a87880cb9975ecd177d` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:51.780098 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3b474-1418-4e37-b140-6c51eea73f71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8e837d350b04a87880cb9975ecd177d)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:51.782468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac3b474-1418-4e37-b140-6c51eea73f71 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:51.868427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cac3b474-1418-4e37-b140-6c51eea73f71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8e837d350b04a87880cb9975ecd177d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:51.885310 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cac3b474-1418-4e37-b140-6c51eea73f71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8e837d350b04a87880cb9975ecd177d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:51.899067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cac3b474-1418-4e37-b140-6c51eea73f71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8e837d350b04a87880cb9975ecd177d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:51.899815 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cac3b474-1418-4e37-b140-6c51eea73f71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8e837d350b04a87880cb9975ecd177d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:51.912325 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cac3b474-1418-4e37-b140-6c51eea73f71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8e837d350b04a87880cb9975ecd177d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:51.927268 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 946/1893] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:51 2022] DELETE /identity/v3/users/f8e837d350b04a87880cb9975ecd177d => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:51.930001 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46f637-fb77-4920-8097-ca3c3e572ee6 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:51.930529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46f637-fb77-4920-8097-ca3c3e572ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:51.931017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46f637-fb77-4920-8097-ca3c3e572ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:51.995260 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-4e46f637-fb77-4920-8097-ca3c3e572ee6 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:51.999631 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4e46f637-fb77-4920-8097-ca3c3e572ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:52.130156 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 948/1894] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:52.143286 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b474-1418-4e37-b140-6c51eea73f71 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:52.145067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-cac3b474-1418-4e37-b140-6c51eea73f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:52.208460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b474-1418-4e37-b140-6c51eea73f71 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:52.212978 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293aaa5c-e6d8-4e55-8db5-ed3655f34d54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:52.213606 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293aaa5c-e6d8-4e55-8db5-ed3655f34d54 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:52.213881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293aaa5c-e6d8-4e55-8db5-ed3655f34d54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:52.214153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293aaa5c-e6d8-4e55-8db5-ed3655f34d54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:52.224032 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293aaa5c-e6d8-4e55-8db5-ed3655f34d54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:52.226471 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293aaa5c-e6d8-4e55-8db5-ed3655f34d54 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:52.236974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 947/1895] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:52.583931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46f637-fb77-4920-8097-ca3c3e572ee6 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:52.585399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4e46f637-fb77-4920-8097-ca3c3e572ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:52.598672 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46f637-fb77-4920-8097-ca3c3e572ee6 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:52.605047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7477ac-92b9-4c22-913b-9f3fef7498c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:52.606076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7477ac-92b9-4c22-913b-9f3fef7498c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:52.606373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7477ac-92b9-4c22-913b-9f3fef7498c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:52.606773 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7477ac-92b9-4c22-913b-9f3fef7498c3 admin admin] PATH_INFO: `/v3/projects/4716c702a68f44939c98995380cb83c2` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:52.608115 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293aaa5c-e6d8-4e55-8db5-ed3655f34d54 service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:52.608383 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7477ac-92b9-4c22-913b-9f3fef7498c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4716c702a68f44939c98995380cb83c2)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:52.609897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-293aaa5c-e6d8-4e55-8db5-ed3655f34d54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:52.610351 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7477ac-92b9-4c22-913b-9f3fef7498c3 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:52.648890 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:52.664499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ed7477ac-92b9-4c22-913b-9f3fef7498c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4716c702a68f44939c98995380cb83c2', 'request_id': 'req-ed7477ac-92b9-4c22-913b-9f3fef7498c3'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:52.671467 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ed7477ac-92b9-4c22-913b-9f3fef7498c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4716c702a68f44939c98995380cb83c2', 'request_id': 'req-ed7477ac-92b9-4c22-913b-9f3fef7498c3'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:52.680659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293aaa5c-e6d8-4e55-8db5-ed3655f34d54 service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:52.681879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ed7477ac-92b9-4c22-913b-9f3fef7498c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4716c702a68f44939c98995380cb83c2', 'request_id': 'req-ed7477ac-92b9-4c22-913b-9f3fef7498c3'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:52.693302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ed7477ac-92b9-4c22-913b-9f3fef7498c3 admin admin] The token cache is being invalidate because project 4716c702a68f44939c98995380cb83c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:52.693601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ed7477ac-92b9-4c22-913b-9f3fef7498c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:52.694281 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ed7477ac-92b9-4c22-913b-9f3fef7498c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:52.695829 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 949/1896] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:52 2022] DELETE /identity/v3/projects/4716c702a68f44939c98995380cb83c2 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:26:52.700096 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7477ac-92b9-4c22-913b-9f3fef7498c3 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:52.701601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ed7477ac-92b9-4c22-913b-9f3fef7498c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:52.718010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:52.737205 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92880d64-b855-46c0-8e96-6ca4d0110877 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:52.737852 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92880d64-b855-46c0-8e96-6ca4d0110877 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:52.738113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92880d64-b855-46c0-8e96-6ca4d0110877 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:52.738399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92880d64-b855-46c0-8e96-6ca4d0110877 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:52.740219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-92880d64-b855-46c0-8e96-6ca4d0110877 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:52.780676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7477ac-92b9-4c22-913b-9f3fef7498c3 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:52.786097 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594b9a8d-d90f-4a30-a3db-bef961a5651d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:52.786756 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b9a8d-d90f-4a30-a3db-bef961a5651d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:52.787044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b9a8d-d90f-4a30-a3db-bef961a5651d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:52.787305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594b9a8d-d90f-4a30-a3db-bef961a5651d admin admin] PATH_INFO: `/v3/users/49715d7f74724d3fbbf7fc9ab8624c5b` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:52.788567 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b9a8d-d90f-4a30-a3db-bef961a5651d admin admin] RBAC: Authorizing `identity:delete_user(user_id=49715d7f74724d3fbbf7fc9ab8624c5b)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:52.790509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594b9a8d-d90f-4a30-a3db-bef961a5651d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:52.813302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92880d64-b855-46c0-8e96-6ca4d0110877 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:52.815058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92880d64-b855-46c0-8e96-6ca4d0110877 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:52.825276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 948/1897] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:52.841964 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-594b9a8d-d90f-4a30-a3db-bef961a5651d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49715d7f74724d3fbbf7fc9ab8624c5b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:52.854588 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-594b9a8d-d90f-4a30-a3db-bef961a5651d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49715d7f74724d3fbbf7fc9ab8624c5b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:52.861900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-594b9a8d-d90f-4a30-a3db-bef961a5651d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49715d7f74724d3fbbf7fc9ab8624c5b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:52.862658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-594b9a8d-d90f-4a30-a3db-bef961a5651d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49715d7f74724d3fbbf7fc9ab8624c5b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:52.863318 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-594b9a8d-d90f-4a30-a3db-bef961a5651d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49715d7f74724d3fbbf7fc9ab8624c5b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:52.881932 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 950/1898] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:52 2022] DELETE /identity/v3/users/49715d7f74724d3fbbf7fc9ab8624c5b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:53.186077 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92880d64-b855-46c0-8e96-6ca4d0110877 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:53.198735 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92880d64-b855-46c0-8e96-6ca4d0110877 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:53.258228 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786a94d-fabf-44ef-a31a-a170cb777708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:53.258941 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786a94d-fabf-44ef-a31a-a170cb777708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:53.259128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786a94d-fabf-44ef-a31a-a170cb777708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:53.259396 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786a94d-fabf-44ef-a31a-a170cb777708 admin admin] PATH_INFO: `/v3/projects/c8c5aa765c7a414d89e4deff5d807fdb` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:53.260530 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e786a94d-fabf-44ef-a31a-a170cb777708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8c5aa765c7a414d89e4deff5d807fdb)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:53.262470 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e786a94d-fabf-44ef-a31a-a170cb777708 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:53.316089 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:53.328114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e786a94d-fabf-44ef-a31a-a170cb777708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8c5aa765c7a414d89e4deff5d807fdb', 'request_id': 'req-e786a94d-fabf-44ef-a31a-a170cb777708'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:53.336821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e786a94d-fabf-44ef-a31a-a170cb777708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8c5aa765c7a414d89e4deff5d807fdb', 'request_id': 'req-e786a94d-fabf-44ef-a31a-a170cb777708'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:53.344659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e786a94d-fabf-44ef-a31a-a170cb777708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8c5aa765c7a414d89e4deff5d807fdb', 'request_id': 'req-e786a94d-fabf-44ef-a31a-a170cb777708'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:53.362051 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e786a94d-fabf-44ef-a31a-a170cb777708 admin admin] The token cache is being invalidate because project c8c5aa765c7a414d89e4deff5d807fdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:53.362345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e786a94d-fabf-44ef-a31a-a170cb777708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:53.363024 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e786a94d-fabf-44ef-a31a-a170cb777708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:53.364665 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 949/1899] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:53 2022] DELETE /identity/v3/projects/c8c5aa765c7a414d89e4deff5d807fdb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:26:53.371059 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ab65b-9885-46af-8cdd-6192abc2a95a None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:53.371667 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ab65b-9885-46af-8cdd-6192abc2a95a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:53.371667 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ab65b-9885-46af-8cdd-6192abc2a95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:53.392402 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-0c2ab65b-9885-46af-8cdd-6192abc2a95a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:53.393673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-0c2ab65b-9885-46af-8cdd-6192abc2a95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:53.470976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 951/1900] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:53.479128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e2e95-d5ed-43ca-8323-6743f2339471 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:53.479430 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e2e95-d5ed-43ca-8323-6743f2339471 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:53.480296 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e2e95-d5ed-43ca-8323-6743f2339471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:53.508271 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-4d0e2e95-d5ed-43ca-8323-6743f2339471 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:53.509589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4d0e2e95-d5ed-43ca-8323-6743f2339471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:53.520797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 950/1901] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:53.522404 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ab65b-9885-46af-8cdd-6192abc2a95a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:53.532743 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ab65b-9885-46af-8cdd-6192abc2a95a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:53.537664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da2792-d184-40aa-823c-209159d05f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:53.538297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da2792-d184-40aa-823c-209159d05f1a admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:53.538579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da2792-d184-40aa-823c-209159d05f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:53.538908 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da2792-d184-40aa-823c-209159d05f1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:53.539525 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da2792-d184-40aa-823c-209159d05f1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:26:53.539798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da2792-d184-40aa-823c-209159d05f1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:53.541459 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da2792-d184-40aa-823c-209159d05f1a admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:53.559294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 952/1902] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:26:53.560243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e2e95-d5ed-43ca-8323-6743f2339471 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:53.572124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e2e95-d5ed-43ca-8323-6743f2339471 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:53.577447 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abab7c-a455-4f85-ac1d-0c425d6ff595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:53.578234 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abab7c-a455-4f85-ac1d-0c425d6ff595 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:53.578517 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abab7c-a455-4f85-ac1d-0c425d6ff595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:53.580446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62abab7c-a455-4f85-ac1d-0c425d6ff595 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:53.581398 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abab7c-a455-4f85-ac1d-0c425d6ff595 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:53.583588 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62abab7c-a455-4f85-ac1d-0c425d6ff595 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:53.616801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 951/1903] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:26:53 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:26:53.621447 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da2792-d184-40aa-823c-209159d05f1a admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:53.632362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da2792-d184-40aa-823c-209159d05f1a admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:53.648752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:53.704072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7da167-cf57-418f-83a6-cb9de4acee14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:53.704760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7da167-cf57-418f-83a6-cb9de4acee14 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:53.705040 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7da167-cf57-418f-83a6-cb9de4acee14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:53.705315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7da167-cf57-418f-83a6-cb9de4acee14 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:53.705949 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7da167-cf57-418f-83a6-cb9de4acee14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:53.707952 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7da167-cf57-418f-83a6-cb9de4acee14 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:53.734467 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 953/1904] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:26:53 2022] POST /identity/v3/users => generated 284 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:26:53.738641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abab7c-a455-4f85-ac1d-0c425d6ff595 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:53.750407 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62abab7c-a455-4f85-ac1d-0c425d6ff595 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:53.756318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7b218-24c0-4d0e-a75a-ee368a045c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:53.757284 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7b218-24c0-4d0e-a75a-ee368a045c77 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:53.757565 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7b218-24c0-4d0e-a75a-ee368a045c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:53.757830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe7b218-24c0-4d0e-a75a-ee368a045c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:53.758457 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7b218-24c0-4d0e-a75a-ee368a045c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:53.760132 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe7b218-24c0-4d0e-a75a-ee368a045c77 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:53.780284 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 952/1905] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:26:53.789214 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7da167-cf57-418f-83a6-cb9de4acee14 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:53.800518 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7da167-cf57-418f-83a6-cb9de4acee14 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:53.805755 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2e8db-3ad5-4400-9f31-e44d75e95523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:53.806375 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2e8db-3ad5-4400-9f31-e44d75e95523 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:53.806777 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2e8db-3ad5-4400-9f31-e44d75e95523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:53.807054 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2e8db-3ad5-4400-9f31-e44d75e95523 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:53.807709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2e8db-3ad5-4400-9f31-e44d75e95523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:53.809368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2e8db-3ad5-4400-9f31-e44d75e95523 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:53.821416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 954/1906] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:26:53.826224 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7b218-24c0-4d0e-a75a-ee368a045c77 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:53.837021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe7b218-24c0-4d0e-a75a-ee368a045c77 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:53.841764 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de8631-8c2c-4894-b853-f0f5d7940256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:53.842425 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de8631-8c2c-4894-b853-f0f5d7940256 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:53.842702 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de8631-8c2c-4894-b853-f0f5d7940256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:53.842975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de8631-8c2c-4894-b853-f0f5d7940256 admin admin] PATH_INFO: `/v3/projects/51c4dbc7057b463b9569e16bd3f86ff4/users/5fafac13941e40e8b1f95f3ac53bb241/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:53.858762 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de8631-8c2c-4894-b853-f0f5d7940256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51c4dbc7057b463b9569e16bd3f86ff4, user_id=5fafac13941e40e8b1f95f3ac53bb241, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:53.861519 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de8631-8c2c-4894-b853-f0f5d7940256 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:53.874756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 953/1907] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:26:53 2022] PUT /identity/v3/projects/51c4dbc7057b463b9569e16bd3f86ff4/users/5fafac13941e40e8b1f95f3ac53bb241/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:53.886418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be801866-8f9f-4490-af2c-5117b07f0aa6 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:53.886890 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be801866-8f9f-4490-af2c-5117b07f0aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:53.887340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be801866-8f9f-4490-af2c-5117b07f0aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:53.919655 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-be801866-8f9f-4490-af2c-5117b07f0aa6 None None] MFA Rules not processed for user `5fafac13941e40e8b1f95f3ac53bb241`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:53.920951 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-be801866-8f9f-4490-af2c-5117b07f0aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:53.989429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 955/1908] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:54.001182 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de8631-8c2c-4894-b853-f0f5d7940256 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:54.003268 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-36de8631-8c2c-4894-b853-f0f5d7940256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:54.087232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de8631-8c2c-4894-b853-f0f5d7940256 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:54.091147 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde3624-cc07-4e5a-b6db-f4cef5663126 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9daa1a3c7654dcfa5ccb03f467ce7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:54.091766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde3624-cc07-4e5a-b6db-f4cef5663126 service glance] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:54.092038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde3624-cc07-4e5a-b6db-f4cef5663126 service glance] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:54.092343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde3624-cc07-4e5a-b6db-f4cef5663126 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:54.101250 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde3624-cc07-4e5a-b6db-f4cef5663126 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:54.102720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde3624-cc07-4e5a-b6db-f4cef5663126 service glance] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:54.112783 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 954/1909] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:55.439076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68db9c-4dff-4af1-9116-b24051fe5ff9 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:55.439711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68db9c-4dff-4af1-9116-b24051fe5ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:55.439711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68db9c-4dff-4af1-9116-b24051fe5ff9 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:55.441262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 956/1910] 10.222.0.17 () {54 vars in 921 bytes} [Sun Aug 7 19:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:26:55.458047 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3937c-9c71-42b5-9b9f-bbf5d0102acc None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:55.458570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3937c-9c71-42b5-9b9f-bbf5d0102acc None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:55.458570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c3937c-9c71-42b5-9b9f-bbf5d0102acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:55.480823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-17c3937c-9c71-42b5-9b9f-bbf5d0102acc None None] MFA Rules not processed for user `a4946aec299f45bd8402cff3159d04d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:55.481962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-17c3937c-9c71-42b5-9b9f-bbf5d0102acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:55.538281 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:55.578339 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 955/1911] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Aug 7 19:26:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:26:55.601257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68db9c-4dff-4af1-9116-b24051fe5ff9 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:55.603102 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5f68db9c-4dff-4af1-9116-b24051fe5ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:55.624192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68db9c-4dff-4af1-9116-b24051fe5ff9 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:55.640717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14fc15-151c-4759-bad2-c76b4ff6875a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:55.641891 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14fc15-151c-4759-bad2-c76b4ff6875a service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:55.642545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14fc15-151c-4759-bad2-c76b4ff6875a service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:55.643179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14fc15-151c-4759-bad2-c76b4ff6875a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:55.666682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14fc15-151c-4759-bad2-c76b4ff6875a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:55.670459 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14fc15-151c-4759-bad2-c76b4ff6875a service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:55.702512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 957/1912] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:56.865299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3937c-9c71-42b5-9b9f-bbf5d0102acc None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:56.866856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-17c3937c-9c71-42b5-9b9f-bbf5d0102acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:56.943730 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c3937c-9c71-42b5-9b9f-bbf5d0102acc None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:56.949235 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1b5e0-ac49-4a3c-86da-87d2bd8457e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:56.949890 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1b5e0-ac49-4a3c-86da-87d2bd8457e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:56.950179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1b5e0-ac49-4a3c-86da-87d2bd8457e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:56.950468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1b5e0-ac49-4a3c-86da-87d2bd8457e6 admin admin] PATH_INFO: `/v3/users/e218c2d173d4466ea8da29bb2371fec1` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:56.951614 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1b5e0-ac49-4a3c-86da-87d2bd8457e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e218c2d173d4466ea8da29bb2371fec1)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:56.953639 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1b5e0-ac49-4a3c-86da-87d2bd8457e6 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:57.004376 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-8fc1b5e0-ac49-4a3c-86da-87d2bd8457e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e218c2d173d4466ea8da29bb2371fec1'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:57.011617 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-8fc1b5e0-ac49-4a3c-86da-87d2bd8457e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e218c2d173d4466ea8da29bb2371fec1'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:57.026298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-8fc1b5e0-ac49-4a3c-86da-87d2bd8457e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e218c2d173d4466ea8da29bb2371fec1'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:57.027242 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-8fc1b5e0-ac49-4a3c-86da-87d2bd8457e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e218c2d173d4466ea8da29bb2371fec1'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:57.036048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-8fc1b5e0-ac49-4a3c-86da-87d2bd8457e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e218c2d173d4466ea8da29bb2371fec1'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:57.042550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 956/1913] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:56 2022] DELETE /identity/v3/users/e218c2d173d4466ea8da29bb2371fec1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:57.047612 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38fc86-beba-4143-b48b-52c18b08690e None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:57.048292 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38fc86-beba-4143-b48b-52c18b08690e None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:57.048292 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38fc86-beba-4143-b48b-52c18b08690e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:57.068385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-dd38fc86-beba-4143-b48b-52c18b08690e None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:26:57.069936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-dd38fc86-beba-4143-b48b-52c18b08690e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:57.137369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 958/1914] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:26:57.147709 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1b5e0-ac49-4a3c-86da-87d2bd8457e6 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:57.149465 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8fc1b5e0-ac49-4a3c-86da-87d2bd8457e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:57.230714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1b5e0-ac49-4a3c-86da-87d2bd8457e6 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:57.234426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89007039-cfc8-438e-b1ce-5478f67eb5f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:57.235147 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89007039-cfc8-438e-b1ce-5478f67eb5f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:57.235522 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89007039-cfc8-438e-b1ce-5478f67eb5f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:57.235831 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89007039-cfc8-438e-b1ce-5478f67eb5f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:57.245966 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89007039-cfc8-438e-b1ce-5478f67eb5f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:57.247437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89007039-cfc8-438e-b1ce-5478f67eb5f6 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:57.257976 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 957/1915] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:26:58.092761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38fc86-beba-4143-b48b-52c18b08690e None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:58.096534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-dd38fc86-beba-4143-b48b-52c18b08690e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:58.111524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38fc86-beba-4143-b48b-52c18b08690e None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:58.130257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:58.132184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:58.133056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:58.133516 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67 admin admin] PATH_INFO: `/v3/projects/91eaae76dd3e43b5817ef44f63fe0329` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:58.151893 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91eaae76dd3e43b5817ef44f63fe0329)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:58.161937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:58.197831 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:58.276893 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91eaae76dd3e43b5817ef44f63fe0329', 'request_id': 'req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:58.288133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91eaae76dd3e43b5817ef44f63fe0329', 'request_id': 'req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:58.303284 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91eaae76dd3e43b5817ef44f63fe0329', 'request_id': 'req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:58.317195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67 admin admin] The token cache is being invalidate because project 91eaae76dd3e43b5817ef44f63fe0329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:58.317515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:58.318187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:58.319981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 959/1916] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:58 2022] DELETE /identity/v3/projects/91eaae76dd3e43b5817ef44f63fe0329 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:58.325359 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89007039-cfc8-438e-b1ce-5478f67eb5f6 service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:58.331618 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:58.339605 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89007039-cfc8-438e-b1ce-5478f67eb5f6 service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:58.403815 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed616a-1925-41f5-9472-7fd90eb7af5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:58.404439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed616a-1925-41f5-9472-7fd90eb7af5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:58.404717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed616a-1925-41f5-9472-7fd90eb7af5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:58.404990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed616a-1925-41f5-9472-7fd90eb7af5c admin admin] PATH_INFO: `/v3/users/d5b987197e8a491a8207a9b521bef86a` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:58.406956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ed616a-1925-41f5-9472-7fd90eb7af5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5b987197e8a491a8207a9b521bef86a)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:58.409630 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ed616a-1925-41f5-9472-7fd90eb7af5c admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:58.481446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-31ed616a-1925-41f5-9472-7fd90eb7af5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5b987197e8a491a8207a9b521bef86a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:58.497328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-31ed616a-1925-41f5-9472-7fd90eb7af5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5b987197e8a491a8207a9b521bef86a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:58.519234 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-31ed616a-1925-41f5-9472-7fd90eb7af5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5b987197e8a491a8207a9b521bef86a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:58.520462 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-31ed616a-1925-41f5-9472-7fd90eb7af5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5b987197e8a491a8207a9b521bef86a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:58.532935 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-31ed616a-1925-41f5-9472-7fd90eb7af5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5b987197e8a491a8207a9b521bef86a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:58.543157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 958/1917] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:58 2022] DELETE /identity/v3/users/d5b987197e8a491a8207a9b521bef86a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:58.935067 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:58.936736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:58.951173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65be1f5-de39-4d9e-aa5a-68f4a7d4ac67 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:59.042969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0d68d-3807-427f-b014-72914e44493e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:59.045855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0d68d-3807-427f-b014-72914e44493e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:59.045855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0d68d-3807-427f-b014-72914e44493e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:59.046062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0d68d-3807-427f-b014-72914e44493e admin admin] PATH_INFO: `/v3/projects/1bb884d16ca645c39918e9c6739e4e92` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:59.049407 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0d68d-3807-427f-b014-72914e44493e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bb884d16ca645c39918e9c6739e4e92)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:59.051830 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0d68d-3807-427f-b014-72914e44493e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:59.205148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-54c0d68d-3807-427f-b014-72914e44493e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bb884d16ca645c39918e9c6739e4e92', 'request_id': 'req-54c0d68d-3807-427f-b014-72914e44493e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:59.218250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-54c0d68d-3807-427f-b014-72914e44493e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bb884d16ca645c39918e9c6739e4e92', 'request_id': 'req-54c0d68d-3807-427f-b014-72914e44493e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:59.240074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-54c0d68d-3807-427f-b014-72914e44493e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bb884d16ca645c39918e9c6739e4e92', 'request_id': 'req-54c0d68d-3807-427f-b014-72914e44493e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:59.267938 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-54c0d68d-3807-427f-b014-72914e44493e admin admin] The token cache is being invalidate because project 1bb884d16ca645c39918e9c6739e4e92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:26:59.269705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-54c0d68d-3807-427f-b014-72914e44493e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:59.270813 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-54c0d68d-3807-427f-b014-72914e44493e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:59.280040 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 960/1918] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:26:58 2022] DELETE /identity/v3/projects/1bb884d16ca645c39918e9c6739e4e92 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:26:59.281212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed616a-1925-41f5-9472-7fd90eb7af5c admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:26:59.283342 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-31ed616a-1925-41f5-9472-7fd90eb7af5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:26:59.322187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed616a-1925-41f5-9472-7fd90eb7af5c admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:26:59.459674 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a35c7-80c6-4df5-8512-f7bcb315d5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:26:59.462140 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a35c7-80c6-4df5-8512-f7bcb315d5fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:26:59.463251 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a35c7-80c6-4df5-8512-f7bcb315d5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:26:59.464022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a35c7-80c6-4df5-8512-f7bcb315d5fb admin admin] PATH_INFO: `/v3/users/4e8fb30a505144ad8685cca92446269a` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:26:59.483183 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a35c7-80c6-4df5-8512-f7bcb315d5fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e8fb30a505144ad8685cca92446269a)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:26:59.487423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a35c7-80c6-4df5-8512-f7bcb315d5fb admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:26:59.528825 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:26:59.584183 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e87a35c7-80c6-4df5-8512-f7bcb315d5fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e8fb30a505144ad8685cca92446269a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:59.596948 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e87a35c7-80c6-4df5-8512-f7bcb315d5fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e8fb30a505144ad8685cca92446269a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:59.623961 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e87a35c7-80c6-4df5-8512-f7bcb315d5fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e8fb30a505144ad8685cca92446269a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:59.625845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e87a35c7-80c6-4df5-8512-f7bcb315d5fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e8fb30a505144ad8685cca92446269a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:59.640142 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e87a35c7-80c6-4df5-8512-f7bcb315d5fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e8fb30a505144ad8685cca92446269a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:26:59.656800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 959/1919] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:26:59 2022] DELETE /identity/v3/users/4e8fb30a505144ad8685cca92446269a => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:27:00.796947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0d68d-3807-427f-b014-72914e44493e admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:00.800444 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-54c0d68d-3807-427f-b014-72914e44493e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:00.837995 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0d68d-3807-427f-b014-72914e44493e admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:00.873120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:00.910434 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e85a6-3385-4d4e-a61f-eda0c7a93c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:00.911094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e85a6-3385-4d4e-a61f-eda0c7a93c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:00.911375 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e85a6-3385-4d4e-a61f-eda0c7a93c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:00.911644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e85a6-3385-4d4e-a61f-eda0c7a93c9d admin admin] PATH_INFO: `/v3/projects/ce4d3f3bce174b3e924d517763b02a9a` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:00.912829 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e85a6-3385-4d4e-a61f-eda0c7a93c9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce4d3f3bce174b3e924d517763b02a9a)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:00.914757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e85a6-3385-4d4e-a61f-eda0c7a93c9d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:00.970625 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-207e85a6-3385-4d4e-a61f-eda0c7a93c9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce4d3f3bce174b3e924d517763b02a9a', 'request_id': 'req-207e85a6-3385-4d4e-a61f-eda0c7a93c9d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:00.976880 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-207e85a6-3385-4d4e-a61f-eda0c7a93c9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce4d3f3bce174b3e924d517763b02a9a', 'request_id': 'req-207e85a6-3385-4d4e-a61f-eda0c7a93c9d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:00.985866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-207e85a6-3385-4d4e-a61f-eda0c7a93c9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce4d3f3bce174b3e924d517763b02a9a', 'request_id': 'req-207e85a6-3385-4d4e-a61f-eda0c7a93c9d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:00.999055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-207e85a6-3385-4d4e-a61f-eda0c7a93c9d admin admin] The token cache is being invalidate because project ce4d3f3bce174b3e924d517763b02a9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:27:00.999251 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-207e85a6-3385-4d4e-a61f-eda0c7a93c9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:00.999970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-207e85a6-3385-4d4e-a61f-eda0c7a93c9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:01.001746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 961/1920] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:27:00 2022] DELETE /identity/v3/projects/ce4d3f3bce174b3e924d517763b02a9a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:01.008885 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2222123-93cd-4d93-b48c-57ada788c813 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:01.010256 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2222123-93cd-4d93-b48c-57ada788c813 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:01.011025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2222123-93cd-4d93-b48c-57ada788c813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:01.035185 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-f2222123-93cd-4d93-b48c-57ada788c813 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:01.037590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f2222123-93cd-4d93-b48c-57ada788c813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:01.112495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 960/1921] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:01.121646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf363b-cf73-4205-96a8-4baa47d59d4f None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:01.121983 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf363b-cf73-4205-96a8-4baa47d59d4f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:01.122653 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf363b-cf73-4205-96a8-4baa47d59d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:01.146745 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-69cf363b-cf73-4205-96a8-4baa47d59d4f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:01.148057 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-69cf363b-cf73-4205-96a8-4baa47d59d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:01.156438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 962/1922] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:01.161261 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2222123-93cd-4d93-b48c-57ada788c813 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:01.172144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2222123-93cd-4d93-b48c-57ada788c813 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:01.177767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef28832-f24c-4672-94d9-47cc9c1089d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:01.178412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef28832-f24c-4672-94d9-47cc9c1089d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:01.178682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef28832-f24c-4672-94d9-47cc9c1089d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:01.178946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef28832-f24c-4672-94d9-47cc9c1089d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:01.179610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef28832-f24c-4672-94d9-47cc9c1089d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:27:01.179911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef28832-f24c-4672-94d9-47cc9c1089d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:01.181640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef28832-f24c-4672-94d9-47cc9c1089d5 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:01.195117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 961/1923] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:27:01.199760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf363b-cf73-4205-96a8-4baa47d59d4f None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:01.210074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf363b-cf73-4205-96a8-4baa47d59d4f None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:01.215641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a8987-cf9f-4a77-8275-da05ed1b9aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:01.216439 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a8987-cf9f-4a77-8275-da05ed1b9aae admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:01.216716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a8987-cf9f-4a77-8275-da05ed1b9aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:01.217162 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a8987-cf9f-4a77-8275-da05ed1b9aae admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:01.217675 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a8987-cf9f-4a77-8275-da05ed1b9aae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:01.219545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a8987-cf9f-4a77-8275-da05ed1b9aae admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:01.259237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 963/1924] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:27:01 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:27:01.264290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef28832-f24c-4672-94d9-47cc9c1089d5 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:01.275851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef28832-f24c-4672-94d9-47cc9c1089d5 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:01.376243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799b679-0e26-4a06-aa6f-355c0e6d2a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:01.376935 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799b679-0e26-4a06-aa6f-355c0e6d2a74 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:01.377210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799b679-0e26-4a06-aa6f-355c0e6d2a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:01.377479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799b679-0e26-4a06-aa6f-355c0e6d2a74 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:01.378109 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799b679-0e26-4a06-aa6f-355c0e6d2a74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:01.380100 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799b679-0e26-4a06-aa6f-355c0e6d2a74 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:01.405304 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 962/1925] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:27:01 2022] POST /identity/v3/users => generated 292 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:01.409989 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a8987-cf9f-4a77-8275-da05ed1b9aae admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:01.423458 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a8987-cf9f-4a77-8275-da05ed1b9aae admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:01.429271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba700d9b-efb2-4b72-9947-e5c3e2393f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:01.429877 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba700d9b-efb2-4b72-9947-e5c3e2393f5b admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:01.430205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba700d9b-efb2-4b72-9947-e5c3e2393f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:01.430441 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba700d9b-efb2-4b72-9947-e5c3e2393f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:01.431111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba700d9b-efb2-4b72-9947-e5c3e2393f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:01.432844 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba700d9b-efb2-4b72-9947-e5c3e2393f5b admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:01.446370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 964/1926] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:01.451315 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799b679-0e26-4a06-aa6f-355c0e6d2a74 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:01.472145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799b679-0e26-4a06-aa6f-355c0e6d2a74 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:01.480908 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b231b-e639-45c9-9730-350f6ff2ef91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:01.481953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b231b-e639-45c9-9730-350f6ff2ef91 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:01.482476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b231b-e639-45c9-9730-350f6ff2ef91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:01.482961 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b231b-e639-45c9-9730-350f6ff2ef91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:01.484189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b231b-e639-45c9-9730-350f6ff2ef91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:01.487406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b231b-e639-45c9-9730-350f6ff2ef91 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:01.502154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 963/1927] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:01.506779 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba700d9b-efb2-4b72-9947-e5c3e2393f5b admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:01.527229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba700d9b-efb2-4b72-9947-e5c3e2393f5b admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:01.533297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13db56d-128a-420e-b8c6-e2c51a6c27b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:01.533992 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13db56d-128a-420e-b8c6-e2c51a6c27b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:01.534272 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13db56d-128a-420e-b8c6-e2c51a6c27b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:01.534547 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13db56d-128a-420e-b8c6-e2c51a6c27b8 admin admin] PATH_INFO: `/v3/projects/bfad12393af54adcb1db64b3d94c18d4/users/901afee892eb4282bbe7cc949203b5a3/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:01.554277 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13db56d-128a-420e-b8c6-e2c51a6c27b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfad12393af54adcb1db64b3d94c18d4, user_id=901afee892eb4282bbe7cc949203b5a3, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:01.557330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13db56d-128a-420e-b8c6-e2c51a6c27b8 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:01.573505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 965/1928] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:01 2022] PUT /identity/v3/projects/bfad12393af54adcb1db64b3d94c18d4/users/901afee892eb4282bbe7cc949203b5a3/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:01.586428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14ce84-3403-40be-963a-32a822ca34d9 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:01.586719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14ce84-3403-40be-963a-32a822ca34d9 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:01.586989 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14ce84-3403-40be-963a-32a822ca34d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:01.601204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:01.619626 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-0c14ce84-3403-40be-963a-32a822ca34d9 None None] MFA Rules not processed for user `901afee892eb4282bbe7cc949203b5a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:01.620913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0c14ce84-3403-40be-963a-32a822ca34d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:01.716057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 964/1929] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:01.718296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13db56d-128a-420e-b8c6-e2c51a6c27b8 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:01.811141 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:01.817442 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13db56d-128a-420e-b8c6-e2c51a6c27b8 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:01.842065 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b769d2d-68fa-4f19-bb2d-a89eae5ae84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:01.843423 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b769d2d-68fa-4f19-bb2d-a89eae5ae84c admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:01.843423 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b769d2d-68fa-4f19-bb2d-a89eae5ae84c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:01.843782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b769d2d-68fa-4f19-bb2d-a89eae5ae84c admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:01.844757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b769d2d-68fa-4f19-bb2d-a89eae5ae84c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:01.847119 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b769d2d-68fa-4f19-bb2d-a89eae5ae84c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:01.880705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 966/1930] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:27:01 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:01.884572 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14ce84-3403-40be-963a-32a822ca34d9 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:01.900352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14ce84-3403-40be-963a-32a822ca34d9 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:01.998173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e5176-5d6f-4dcf-b5ca-2bdbb0279e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:01.999122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e5176-5d6f-4dcf-b5ca-2bdbb0279e8d admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:01.999516 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e5176-5d6f-4dcf-b5ca-2bdbb0279e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:01.999907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e5176-5d6f-4dcf-b5ca-2bdbb0279e8d admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:02.000701 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e5176-5d6f-4dcf-b5ca-2bdbb0279e8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:02.002808 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e5176-5d6f-4dcf-b5ca-2bdbb0279e8d admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:02.029838 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 965/1931] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:27:01 2022] POST /identity/v3/users => generated 298 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:27:02.031507 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b769d2d-68fa-4f19-bb2d-a89eae5ae84c admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:02.042645 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b769d2d-68fa-4f19-bb2d-a89eae5ae84c admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:02.047865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a785d9-8de5-4476-b1f0-ac7c9398a31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:02.048554 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a785d9-8de5-4476-b1f0-ac7c9398a31d admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:02.048837 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a785d9-8de5-4476-b1f0-ac7c9398a31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:02.049251 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a785d9-8de5-4476-b1f0-ac7c9398a31d admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:02.049918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a785d9-8de5-4476-b1f0-ac7c9398a31d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:02.051545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a785d9-8de5-4476-b1f0-ac7c9398a31d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:02.063344 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 967/1932] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:02.067512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e5176-5d6f-4dcf-b5ca-2bdbb0279e8d admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:02.079898 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e5176-5d6f-4dcf-b5ca-2bdbb0279e8d admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:02.084982 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760db74d-9a65-4bca-bea1-7be7384a9e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:02.085709 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760db74d-9a65-4bca-bea1-7be7384a9e11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:02.086072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760db74d-9a65-4bca-bea1-7be7384a9e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:02.086599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760db74d-9a65-4bca-bea1-7be7384a9e11 admin admin] PATH_INFO: `/v3/projects/2ee4932680db43c38d1f36ac004637fc/users/348ccabde042445384d3fcd371f4c8d0/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:02.107932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760db74d-9a65-4bca-bea1-7be7384a9e11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee4932680db43c38d1f36ac004637fc, user_id=348ccabde042445384d3fcd371f4c8d0, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:02.111407 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760db74d-9a65-4bca-bea1-7be7384a9e11 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:02.130701 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 966/1933] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:02 2022] PUT /identity/v3/projects/2ee4932680db43c38d1f36ac004637fc/users/348ccabde042445384d3fcd371f4c8d0/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:02.135968 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a785d9-8de5-4476-b1f0-ac7c9398a31d admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:02.153598 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a785d9-8de5-4476-b1f0-ac7c9398a31d admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:02.235681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2962b49-e7a3-4f5b-8e17-ae56e022155a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:02.236369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2962b49-e7a3-4f5b-8e17-ae56e022155a admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:02.236649 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2962b49-e7a3-4f5b-8e17-ae56e022155a admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:02.237461 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2962b49-e7a3-4f5b-8e17-ae56e022155a admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:02.237724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2962b49-e7a3-4f5b-8e17-ae56e022155a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:02.239429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2962b49-e7a3-4f5b-8e17-ae56e022155a admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:02.250446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 968/1934] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:02.254899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760db74d-9a65-4bca-bea1-7be7384a9e11 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:02.266570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760db74d-9a65-4bca-bea1-7be7384a9e11 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:02.273425 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac333a88-8ea7-48b7-a6cd-e06ba3f26afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:02.274202 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac333a88-8ea7-48b7-a6cd-e06ba3f26afc admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:02.274541 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac333a88-8ea7-48b7-a6cd-e06ba3f26afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:02.274876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac333a88-8ea7-48b7-a6cd-e06ba3f26afc admin admin] PATH_INFO: `/v3/projects/2ee4932680db43c38d1f36ac004637fc/users/348ccabde042445384d3fcd371f4c8d0/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:02.277421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac333a88-8ea7-48b7-a6cd-e06ba3f26afc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee4932680db43c38d1f36ac004637fc, user_id=348ccabde042445384d3fcd371f4c8d0, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:02.280272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac333a88-8ea7-48b7-a6cd-e06ba3f26afc admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:02.296771 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 967/1935] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:02 2022] PUT /identity/v3/projects/2ee4932680db43c38d1f36ac004637fc/users/348ccabde042445384d3fcd371f4c8d0/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:02.308729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe827e0-9e92-4a7c-989b-60dfbf565411 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:02.309087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe827e0-9e92-4a7c-989b-60dfbf565411 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:02.309430 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe827e0-9e92-4a7c-989b-60dfbf565411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:02.345956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-bbe827e0-9e92-4a7c-989b-60dfbf565411 None None] MFA Rules not processed for user `348ccabde042445384d3fcd371f4c8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:02.348139 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-bbe827e0-9e92-4a7c-989b-60dfbf565411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:02.439277 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 969/1936] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:02.450677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac333a88-8ea7-48b7-a6cd-e06ba3f26afc admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:02.452690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-ac333a88-8ea7-48b7-a6cd-e06ba3f26afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:02.522509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:02.527050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac333a88-8ea7-48b7-a6cd-e06ba3f26afc admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:02.531741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb874f-c230-4822-be23-facbe1958efc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:02.532470 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb874f-c230-4822-be23-facbe1958efc service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:02.532910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb874f-c230-4822-be23-facbe1958efc service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:02.533362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb874f-c230-4822-be23-facbe1958efc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:02.543196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb874f-c230-4822-be23-facbe1958efc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:02.544716 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb874f-c230-4822-be23-facbe1958efc service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:02.555842 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 968/1937] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:02.668076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe827e0-9e92-4a7c-989b-60dfbf565411 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:02.678680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe827e0-9e92-4a7c-989b-60dfbf565411 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:02.685113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7cb47-da57-4946-a06a-4e9c9f72773d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:02.685861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7cb47-da57-4946-a06a-4e9c9f72773d service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:02.686245 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7cb47-da57-4946-a06a-4e9c9f72773d service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:02.686500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7cb47-da57-4946-a06a-4e9c9f72773d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:02.745262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:02.767708 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f7cb47-da57-4946-a06a-4e9c9f72773d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:02.769497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f7cb47-da57-4946-a06a-4e9c9f72773d service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:02.785411 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 970/1938] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:05.884919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb874f-c230-4822-be23-facbe1958efc service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:05.963235 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb874f-c230-4822-be23-facbe1958efc service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:05.969897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cf18e-6dc3-447f-aedd-bd5a0dbfab75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:05.970680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cf18e-6dc3-447f-aedd-bd5a0dbfab75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:05.971087 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cf18e-6dc3-447f-aedd-bd5a0dbfab75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:05.971475 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cf18e-6dc3-447f-aedd-bd5a0dbfab75 admin admin] PATH_INFO: `/v3/users/901afee892eb4282bbe7cc949203b5a3` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:05.972887 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1cf18e-6dc3-447f-aedd-bd5a0dbfab75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=901afee892eb4282bbe7cc949203b5a3)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:05.974856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1cf18e-6dc3-447f-aedd-bd5a0dbfab75 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:06.044487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2f1cf18e-6dc3-447f-aedd-bd5a0dbfab75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '901afee892eb4282bbe7cc949203b5a3'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:06.052998 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2f1cf18e-6dc3-447f-aedd-bd5a0dbfab75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '901afee892eb4282bbe7cc949203b5a3'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:06.066168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2f1cf18e-6dc3-447f-aedd-bd5a0dbfab75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '901afee892eb4282bbe7cc949203b5a3'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:06.066907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2f1cf18e-6dc3-447f-aedd-bd5a0dbfab75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '901afee892eb4282bbe7cc949203b5a3'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:06.076740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2f1cf18e-6dc3-447f-aedd-bd5a0dbfab75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '901afee892eb4282bbe7cc949203b5a3'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:06.085048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 969/1939] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:27:05 2022] DELETE /identity/v3/users/901afee892eb4282bbe7cc949203b5a3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:06.089865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14781b21-a6f4-4e98-863a-23fd10822ed6 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:06.090431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14781b21-a6f4-4e98-863a-23fd10822ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:06.090431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14781b21-a6f4-4e98-863a-23fd10822ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:06.111510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-14781b21-a6f4-4e98-863a-23fd10822ed6 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:06.112779 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-14781b21-a6f4-4e98-863a-23fd10822ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:06.205056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 971/1940] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:06.211510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cf18e-6dc3-447f-aedd-bd5a0dbfab75 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:06.212983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-2f1cf18e-6dc3-447f-aedd-bd5a0dbfab75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:06.283540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cf18e-6dc3-447f-aedd-bd5a0dbfab75 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:06.287420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edd760-8d4c-4a88-8ccd-67c99d3bd33d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:06.288204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edd760-8d4c-4a88-8ccd-67c99d3bd33d service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:06.288479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edd760-8d4c-4a88-8ccd-67c99d3bd33d service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:06.288903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edd760-8d4c-4a88-8ccd-67c99d3bd33d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:06.298914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edd760-8d4c-4a88-8ccd-67c99d3bd33d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:06.300432 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edd760-8d4c-4a88-8ccd-67c99d3bd33d service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:06.312446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 970/1941] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:06.638192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14781b21-a6f4-4e98-863a-23fd10822ed6 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:06.640234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-14781b21-a6f4-4e98-863a-23fd10822ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:06.653567 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14781b21-a6f4-4e98-863a-23fd10822ed6 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:06.665879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ace752-563a-4cf0-931c-c5c548d70e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:06.666535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ace752-563a-4cf0-931c-c5c548d70e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:06.666811 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ace752-563a-4cf0-931c-c5c548d70e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:06.667069 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ace752-563a-4cf0-931c-c5c548d70e8c admin admin] PATH_INFO: `/v3/projects/bfad12393af54adcb1db64b3d94c18d4` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:06.668303 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ace752-563a-4cf0-931c-c5c548d70e8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfad12393af54adcb1db64b3d94c18d4)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:06.670126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ace752-563a-4cf0-931c-c5c548d70e8c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:06.721781 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-77ace752-563a-4cf0-931c-c5c548d70e8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfad12393af54adcb1db64b3d94c18d4', 'request_id': 'req-77ace752-563a-4cf0-931c-c5c548d70e8c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:06.728128 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-77ace752-563a-4cf0-931c-c5c548d70e8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfad12393af54adcb1db64b3d94c18d4', 'request_id': 'req-77ace752-563a-4cf0-931c-c5c548d70e8c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:06.737556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-77ace752-563a-4cf0-931c-c5c548d70e8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfad12393af54adcb1db64b3d94c18d4', 'request_id': 'req-77ace752-563a-4cf0-931c-c5c548d70e8c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:06.756285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-77ace752-563a-4cf0-931c-c5c548d70e8c admin admin] The token cache is being invalidate because project bfad12393af54adcb1db64b3d94c18d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:27:06.756750 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-77ace752-563a-4cf0-931c-c5c548d70e8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:06.757795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-77ace752-563a-4cf0-931c-c5c548d70e8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:06.760564 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 972/1942] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:27:06 2022] DELETE /identity/v3/projects/bfad12393af54adcb1db64b3d94c18d4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:06.766083 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edd760-8d4c-4a88-8ccd-67c99d3bd33d service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:06.778604 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edd760-8d4c-4a88-8ccd-67c99d3bd33d service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:06.791275 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:06.848372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac2b88-d67d-422e-8912-18c0c0c18407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:06.849013 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac2b88-d67d-422e-8912-18c0c0c18407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:06.849292 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac2b88-d67d-422e-8912-18c0c0c18407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:06.849559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac2b88-d67d-422e-8912-18c0c0c18407 admin admin] PATH_INFO: `/v3/users/348ccabde042445384d3fcd371f4c8d0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:06.850713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac2b88-d67d-422e-8912-18c0c0c18407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=348ccabde042445384d3fcd371f4c8d0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:06.852687 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac2b88-d67d-422e-8912-18c0c0c18407 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:06.905862 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-44ac2b88-d67d-422e-8912-18c0c0c18407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '348ccabde042445384d3fcd371f4c8d0'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:06.915539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-44ac2b88-d67d-422e-8912-18c0c0c18407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '348ccabde042445384d3fcd371f4c8d0'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:06.930941 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-44ac2b88-d67d-422e-8912-18c0c0c18407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '348ccabde042445384d3fcd371f4c8d0'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:06.931814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-44ac2b88-d67d-422e-8912-18c0c0c18407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '348ccabde042445384d3fcd371f4c8d0'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:06.942078 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-44ac2b88-d67d-422e-8912-18c0c0c18407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '348ccabde042445384d3fcd371f4c8d0'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:06.949498 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 971/1943] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:27:06 2022] DELETE /identity/v3/users/348ccabde042445384d3fcd371f4c8d0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:07.277974 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ace752-563a-4cf0-931c-c5c548d70e8c admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:07.279538 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-77ace752-563a-4cf0-931c-c5c548d70e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:07.294938 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ace752-563a-4cf0-931c-c5c548d70e8c admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:07.355605 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749376e-cccc-4143-aa48-ac0a0ac20b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:07.356296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749376e-cccc-4143-aa48-ac0a0ac20b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:07.356563 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749376e-cccc-4143-aa48-ac0a0ac20b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:07.356832 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749376e-cccc-4143-aa48-ac0a0ac20b07 admin admin] PATH_INFO: `/v3/projects/2ee4932680db43c38d1f36ac004637fc` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:07.357957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749376e-cccc-4143-aa48-ac0a0ac20b07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ee4932680db43c38d1f36ac004637fc)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:07.359909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749376e-cccc-4143-aa48-ac0a0ac20b07 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:07.371474 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:07.411259 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5749376e-cccc-4143-aa48-ac0a0ac20b07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ee4932680db43c38d1f36ac004637fc', 'request_id': 'req-5749376e-cccc-4143-aa48-ac0a0ac20b07'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:07.417687 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5749376e-cccc-4143-aa48-ac0a0ac20b07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ee4932680db43c38d1f36ac004637fc', 'request_id': 'req-5749376e-cccc-4143-aa48-ac0a0ac20b07'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:07.426228 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5749376e-cccc-4143-aa48-ac0a0ac20b07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ee4932680db43c38d1f36ac004637fc', 'request_id': 'req-5749376e-cccc-4143-aa48-ac0a0ac20b07'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:07.436805 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5749376e-cccc-4143-aa48-ac0a0ac20b07 admin admin] The token cache is being invalidate because project 2ee4932680db43c38d1f36ac004637fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:27:07.437099 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5749376e-cccc-4143-aa48-ac0a0ac20b07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:07.437732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5749376e-cccc-4143-aa48-ac0a0ac20b07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:07.439253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 973/1944] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:27:07 2022] DELETE /identity/v3/projects/2ee4932680db43c38d1f36ac004637fc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:07.444365 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83390fbe-aab6-4ab2-b7b9-2f28505573ad None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:07.444581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83390fbe-aab6-4ab2-b7b9-2f28505573ad None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:07.444853 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83390fbe-aab6-4ab2-b7b9-2f28505573ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:07.465095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-83390fbe-aab6-4ab2-b7b9-2f28505573ad None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:07.466302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-83390fbe-aab6-4ab2-b7b9-2f28505573ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:07.528460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 972/1945] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:07.537871 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4b498-4412-4334-80c3-d1eac686b5fc None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:07.538162 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4b498-4412-4334-80c3-d1eac686b5fc None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:07.541248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4b498-4412-4334-80c3-d1eac686b5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:07.564656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-69e4b498-4412-4334-80c3-d1eac686b5fc None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:07.565895 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-69e4b498-4412-4334-80c3-d1eac686b5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:07.575326 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 974/1946] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:07.579947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83390fbe-aab6-4ab2-b7b9-2f28505573ad None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:07.590220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83390fbe-aab6-4ab2-b7b9-2f28505573ad None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:07.596508 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc069f1e-6e62-4811-923c-611982a5d576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:07.597132 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc069f1e-6e62-4811-923c-611982a5d576 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:07.597408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc069f1e-6e62-4811-923c-611982a5d576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:07.597672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc069f1e-6e62-4811-923c-611982a5d576 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:07.598327 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc069f1e-6e62-4811-923c-611982a5d576 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:27:07.598625 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc069f1e-6e62-4811-923c-611982a5d576 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:07.600262 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc069f1e-6e62-4811-923c-611982a5d576 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:07.613825 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 973/1947] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:27:07.618363 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4b498-4412-4334-80c3-d1eac686b5fc None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:07.627952 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4b498-4412-4334-80c3-d1eac686b5fc None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:07.632918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19421d19-8c40-4160-ae10-161dd5845cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:07.633422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19421d19-8c40-4160-ae10-161dd5845cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:07.633779 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19421d19-8c40-4160-ae10-161dd5845cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:07.634098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19421d19-8c40-4160-ae10-161dd5845cc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:07.634710 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19421d19-8c40-4160-ae10-161dd5845cc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:07.636639 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19421d19-8c40-4160-ae10-161dd5845cc8 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:07.666248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 975/1948] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:27:07 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:07.670499 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc069f1e-6e62-4811-923c-611982a5d576 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:07.681736 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc069f1e-6e62-4811-923c-611982a5d576 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:07.700761 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:07.763071 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e01bd0-bdd9-4946-801b-0d475fe361b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:07.763690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e01bd0-bdd9-4946-801b-0d475fe361b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:07.763967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e01bd0-bdd9-4946-801b-0d475fe361b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:07.764475 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e01bd0-bdd9-4946-801b-0d475fe361b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:07.765176 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e01bd0-bdd9-4946-801b-0d475fe361b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:07.767481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e01bd0-bdd9-4946-801b-0d475fe361b7 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:07.789821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 974/1949] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:27:07 2022] POST /identity/v3/users => generated 286 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:07.794313 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19421d19-8c40-4160-ae10-161dd5845cc8 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:07.805899 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19421d19-8c40-4160-ae10-161dd5845cc8 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:07.810930 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19dc97-d597-47c6-bc1d-f6c15e513c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:07.811758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19dc97-d597-47c6-bc1d-f6c15e513c93 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:07.812071 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19dc97-d597-47c6-bc1d-f6c15e513c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:07.812340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19dc97-d597-47c6-bc1d-f6c15e513c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:07.812992 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19dc97-d597-47c6-bc1d-f6c15e513c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:07.814815 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19dc97-d597-47c6-bc1d-f6c15e513c93 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:07.826940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 976/1950] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:07.830995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e01bd0-bdd9-4946-801b-0d475fe361b7 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:07.842033 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e01bd0-bdd9-4946-801b-0d475fe361b7 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:07.847609 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbe654-5f0d-4018-8da5-a6b4b8a2fc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:07.848283 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbe654-5f0d-4018-8da5-a6b4b8a2fc8d admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:07.848556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbe654-5f0d-4018-8da5-a6b4b8a2fc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:07.848824 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbe654-5f0d-4018-8da5-a6b4b8a2fc8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:07.849472 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbe654-5f0d-4018-8da5-a6b4b8a2fc8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:07.851126 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbe654-5f0d-4018-8da5-a6b4b8a2fc8d admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:07.864637 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 975/1951] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:07.868228 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19dc97-d597-47c6-bc1d-f6c15e513c93 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:07.881046 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19dc97-d597-47c6-bc1d-f6c15e513c93 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:07.886602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be65a8-88b2-4c55-abb2-d451e76b5485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:07.887270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be65a8-88b2-4c55-abb2-d451e76b5485 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:07.887544 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be65a8-88b2-4c55-abb2-d451e76b5485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:07.887814 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be65a8-88b2-4c55-abb2-d451e76b5485 admin admin] PATH_INFO: `/v3/projects/8df3773770a54a629538b2cf6d70b8d4/users/a02a5e8b67c94800a6f0f3604366f80a/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:07.905721 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be65a8-88b2-4c55-abb2-d451e76b5485 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8df3773770a54a629538b2cf6d70b8d4, user_id=a02a5e8b67c94800a6f0f3604366f80a, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:07.908498 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be65a8-88b2-4c55-abb2-d451e76b5485 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:07.926705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 977/1952] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:07 2022] PUT /identity/v3/projects/8df3773770a54a629538b2cf6d70b8d4/users/a02a5e8b67c94800a6f0f3604366f80a/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:07.931914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8615689-fd84-444e-8c31-5a0e88c0383a None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:07.932428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8615689-fd84-444e-8c31-5a0e88c0383a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:07.932514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8615689-fd84-444e-8c31-5a0e88c0383a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:07.954050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-c8615689-fd84-444e-8c31-5a0e88c0383a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:07.955209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c8615689-fd84-444e-8c31-5a0e88c0383a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:08.024674 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 976/1953] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:08.035684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be65a8-88b2-4c55-abb2-d451e76b5485 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:08.037417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f0be65a8-88b2-4c55-abb2-d451e76b5485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:08.105337 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be65a8-88b2-4c55-abb2-d451e76b5485 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:08.109126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230efe41-17d6-4efd-991e-214edceee82d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:08.109828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230efe41-17d6-4efd-991e-214edceee82d service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:08.110107 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230efe41-17d6-4efd-991e-214edceee82d service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:08.110526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230efe41-17d6-4efd-991e-214edceee82d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:08.121257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230efe41-17d6-4efd-991e-214edceee82d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:08.122710 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230efe41-17d6-4efd-991e-214edceee82d service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:08.133020 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 978/1954] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:09.011822 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e545a93-e374-4baa-842b-914061a9dc78 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:09.013576 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e545a93-e374-4baa-842b-914061a9dc78 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:09.014049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e545a93-e374-4baa-842b-914061a9dc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:09.048687 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-8e545a93-e374-4baa-842b-914061a9dc78 None None] MFA Rules not processed for user `a02a5e8b67c94800a6f0f3604366f80a`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:09.050249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8e545a93-e374-4baa-842b-914061a9dc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:09.132850 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 977/1955] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:09.137639 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230efe41-17d6-4efd-991e-214edceee82d service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:09.147510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230efe41-17d6-4efd-991e-214edceee82d service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:09.152084 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d33954-da17-49d3-aeac-58530dc2ef17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:09.152700 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d33954-da17-49d3-aeac-58530dc2ef17 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:09.152981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d33954-da17-49d3-aeac-58530dc2ef17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:09.153255 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d33954-da17-49d3-aeac-58530dc2ef17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:09.153891 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d33954-da17-49d3-aeac-58530dc2ef17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:09.155791 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d33954-da17-49d3-aeac-58530dc2ef17 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:09.184533 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 979/1956] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:27:09 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:09.189139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e545a93-e374-4baa-842b-914061a9dc78 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:09.200266 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e545a93-e374-4baa-842b-914061a9dc78 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:09.226172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:09.259308 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92202393-78c8-46ad-9a3f-1ac9d97dc812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:09.260108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92202393-78c8-46ad-9a3f-1ac9d97dc812 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:09.260388 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92202393-78c8-46ad-9a3f-1ac9d97dc812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:09.260656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92202393-78c8-46ad-9a3f-1ac9d97dc812 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:09.261324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92202393-78c8-46ad-9a3f-1ac9d97dc812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:09.264086 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92202393-78c8-46ad-9a3f-1ac9d97dc812 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:09.286800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 978/1957] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:27:09 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:09.290959 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d33954-da17-49d3-aeac-58530dc2ef17 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:09.301908 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d33954-da17-49d3-aeac-58530dc2ef17 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:09.307255 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42332985-1bdb-4359-b6b5-c30a563b5ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:09.307925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42332985-1bdb-4359-b6b5-c30a563b5ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:09.308218 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42332985-1bdb-4359-b6b5-c30a563b5ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:09.308489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42332985-1bdb-4359-b6b5-c30a563b5ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:09.309153 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42332985-1bdb-4359-b6b5-c30a563b5ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:09.310769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42332985-1bdb-4359-b6b5-c30a563b5ba2 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:09.324028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 980/1958] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:09.326658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92202393-78c8-46ad-9a3f-1ac9d97dc812 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:09.337489 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92202393-78c8-46ad-9a3f-1ac9d97dc812 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:09.343618 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08abe2b9-9698-4ae9-a799-3eea7d9dd534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:09.344555 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abe2b9-9698-4ae9-a799-3eea7d9dd534 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:09.344878 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abe2b9-9698-4ae9-a799-3eea7d9dd534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:09.345155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abe2b9-9698-4ae9-a799-3eea7d9dd534 admin admin] PATH_INFO: `/v3/projects/210f612935e743f3a41d0c1a2bb986ca/users/02182da4694b4a1aa2b6a0b8b1013971/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:09.361736 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08abe2b9-9698-4ae9-a799-3eea7d9dd534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=210f612935e743f3a41d0c1a2bb986ca, user_id=02182da4694b4a1aa2b6a0b8b1013971, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:09.364506 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08abe2b9-9698-4ae9-a799-3eea7d9dd534 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:09.383248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 979/1959] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:09 2022] PUT /identity/v3/projects/210f612935e743f3a41d0c1a2bb986ca/users/02182da4694b4a1aa2b6a0b8b1013971/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:27:09.387281 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42332985-1bdb-4359-b6b5-c30a563b5ba2 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:09.398049 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42332985-1bdb-4359-b6b5-c30a563b5ba2 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:09.408476 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:09.471662 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf2c97-3275-4312-a0ff-40661641c0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:09.472475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf2c97-3275-4312-a0ff-40661641c0bf admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:09.472614 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf2c97-3275-4312-a0ff-40661641c0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:09.472886 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf2c97-3275-4312-a0ff-40661641c0bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:09.473705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf2c97-3275-4312-a0ff-40661641c0bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:09.475365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf2c97-3275-4312-a0ff-40661641c0bf admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:09.486071 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 981/1960] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:09.490277 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08abe2b9-9698-4ae9-a799-3eea7d9dd534 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:09.501243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08abe2b9-9698-4ae9-a799-3eea7d9dd534 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:09.505944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef85743-ad65-48a9-9ff2-4a97beb9d953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:09.506614 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef85743-ad65-48a9-9ff2-4a97beb9d953 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:09.506891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef85743-ad65-48a9-9ff2-4a97beb9d953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:09.507160 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef85743-ad65-48a9-9ff2-4a97beb9d953 admin admin] PATH_INFO: `/v3/projects/210f612935e743f3a41d0c1a2bb986ca/users/02182da4694b4a1aa2b6a0b8b1013971/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:09.509102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef85743-ad65-48a9-9ff2-4a97beb9d953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=210f612935e743f3a41d0c1a2bb986ca, user_id=02182da4694b4a1aa2b6a0b8b1013971, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:09.511765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef85743-ad65-48a9-9ff2-4a97beb9d953 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:09.522537 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 980/1961] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:09 2022] PUT /identity/v3/projects/210f612935e743f3a41d0c1a2bb986ca/users/02182da4694b4a1aa2b6a0b8b1013971/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:10.192341 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d067caa-8185-4390-b10e-7aa0f2dfe537 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:10.192921 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d067caa-8185-4390-b10e-7aa0f2dfe537 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:10.192921 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d067caa-8185-4390-b10e-7aa0f2dfe537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:10.230122 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-9d067caa-8185-4390-b10e-7aa0f2dfe537 None None] MFA Rules not processed for user `02182da4694b4a1aa2b6a0b8b1013971`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:10.231379 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9d067caa-8185-4390-b10e-7aa0f2dfe537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:10.320295 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 982/1962] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:10.336862 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef85743-ad65-48a9-9ff2-4a97beb9d953 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:10.339518 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8ef85743-ad65-48a9-9ff2-4a97beb9d953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:10.404344 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef85743-ad65-48a9-9ff2-4a97beb9d953 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:10.410758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a6ac7-eb44-4505-b036-5f2fe785cda7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:10.411478 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a6ac7-eb44-4505-b036-5f2fe785cda7 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:10.412238 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a6ac7-eb44-4505-b036-5f2fe785cda7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:10.412548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1a6ac7-eb44-4505-b036-5f2fe785cda7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:10.480578 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a6ac7-eb44-4505-b036-5f2fe785cda7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:10.482163 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1a6ac7-eb44-4505-b036-5f2fe785cda7 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:10.492211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 981/1963] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:27:12.917269 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d067caa-8185-4390-b10e-7aa0f2dfe537 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:12.930423 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d067caa-8185-4390-b10e-7aa0f2dfe537 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:12.935129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5e004-6792-420d-b7bf-a6eee1de81d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:12.935798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5e004-6792-420d-b7bf-a6eee1de81d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:12.936151 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5e004-6792-420d-b7bf-a6eee1de81d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:12.936444 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5e004-6792-420d-b7bf-a6eee1de81d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:12.948097 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5e004-6792-420d-b7bf-a6eee1de81d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:12.949614 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e5e004-6792-420d-b7bf-a6eee1de81d4 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:12.973131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 983/1964] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:13.519330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a6ac7-eb44-4505-b036-5f2fe785cda7 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:13.520805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-bd1a6ac7-eb44-4505-b036-5f2fe785cda7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:13.565794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:13.591898 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1a6ac7-eb44-4505-b036-5f2fe785cda7 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:13.597364 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f823b08-09b7-4c33-a4ed-13f81e0ca045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:13.598168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f823b08-09b7-4c33-a4ed-13f81e0ca045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:13.598467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f823b08-09b7-4c33-a4ed-13f81e0ca045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:13.598740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f823b08-09b7-4c33-a4ed-13f81e0ca045 admin admin] PATH_INFO: `/v3/users/a4a1ef092eeb493886ad12f3bedca097` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:13.599958 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f823b08-09b7-4c33-a4ed-13f81e0ca045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4a1ef092eeb493886ad12f3bedca097)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:13.602970 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f823b08-09b7-4c33-a4ed-13f81e0ca045 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:13.673004 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3f823b08-09b7-4c33-a4ed-13f81e0ca045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4a1ef092eeb493886ad12f3bedca097'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:13.682280 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3f823b08-09b7-4c33-a4ed-13f81e0ca045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4a1ef092eeb493886ad12f3bedca097'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:13.697562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3f823b08-09b7-4c33-a4ed-13f81e0ca045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4a1ef092eeb493886ad12f3bedca097'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:13.698379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3f823b08-09b7-4c33-a4ed-13f81e0ca045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4a1ef092eeb493886ad12f3bedca097'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:13.706814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3f823b08-09b7-4c33-a4ed-13f81e0ca045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4a1ef092eeb493886ad12f3bedca097'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:13.716084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 982/1965] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:27:13 2022] DELETE /identity/v3/users/a4a1ef092eeb493886ad12f3bedca097 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:13.720129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13269b03-1156-407f-bb0f-cf928ca9b121 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:13.720422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13269b03-1156-407f-bb0f-cf928ca9b121 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:13.720847 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13269b03-1156-407f-bb0f-cf928ca9b121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:13.745876 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-13269b03-1156-407f-bb0f-cf928ca9b121 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:13.747143 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-13269b03-1156-407f-bb0f-cf928ca9b121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:13.823261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 984/1966] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:13.833208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f823b08-09b7-4c33-a4ed-13f81e0ca045 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:13.834621 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3f823b08-09b7-4c33-a4ed-13f81e0ca045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:13.897829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f823b08-09b7-4c33-a4ed-13f81e0ca045 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:13.901949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b4f11-998f-4557-973f-df663ac025e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:13.902574 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b4f11-998f-4557-973f-df663ac025e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:13.902878 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b4f11-998f-4557-973f-df663ac025e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:13.903243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b4f11-998f-4557-973f-df663ac025e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:13.913454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b4f11-998f-4557-973f-df663ac025e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:13.914916 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b4f11-998f-4557-973f-df663ac025e2 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:13.926466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 983/1967] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:14.430756 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13269b03-1156-407f-bb0f-cf928ca9b121 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:14.432344 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-13269b03-1156-407f-bb0f-cf928ca9b121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:14.460380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13269b03-1156-407f-bb0f-cf928ca9b121 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:14.474208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d16e1-c28f-48ac-a6da-93d8efc230c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:14.474959 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d16e1-c28f-48ac-a6da-93d8efc230c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:14.476222 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d16e1-c28f-48ac-a6da-93d8efc230c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:14.476846 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d16e1-c28f-48ac-a6da-93d8efc230c6 admin admin] PATH_INFO: `/v3/projects/7c663e9a8ec74db2aa769769fcb04118` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:14.478528 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d16e1-c28f-48ac-a6da-93d8efc230c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c663e9a8ec74db2aa769769fcb04118)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:14.481466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d16e1-c28f-48ac-a6da-93d8efc230c6 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:14.497267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:14.550448 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-734d16e1-c28f-48ac-a6da-93d8efc230c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c663e9a8ec74db2aa769769fcb04118', 'request_id': 'req-734d16e1-c28f-48ac-a6da-93d8efc230c6'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:14.559452 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-734d16e1-c28f-48ac-a6da-93d8efc230c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c663e9a8ec74db2aa769769fcb04118', 'request_id': 'req-734d16e1-c28f-48ac-a6da-93d8efc230c6'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:14.570013 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-734d16e1-c28f-48ac-a6da-93d8efc230c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c663e9a8ec74db2aa769769fcb04118', 'request_id': 'req-734d16e1-c28f-48ac-a6da-93d8efc230c6'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:14.587870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-734d16e1-c28f-48ac-a6da-93d8efc230c6 admin admin] The token cache is being invalidate because project 7c663e9a8ec74db2aa769769fcb04118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:27:14.588614 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-734d16e1-c28f-48ac-a6da-93d8efc230c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:14.589570 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-734d16e1-c28f-48ac-a6da-93d8efc230c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:14.595885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 985/1968] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:27:14 2022] DELETE /identity/v3/projects/7c663e9a8ec74db2aa769769fcb04118 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:14.597357 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360891a-ffac-4ebf-8c59-1e496d6905e6 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:14.597966 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360891a-ffac-4ebf-8c59-1e496d6905e6 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:14.598071 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f360891a-ffac-4ebf-8c59-1e496d6905e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:14.635686 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-f360891a-ffac-4ebf-8c59-1e496d6905e6 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:14.638999 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f360891a-ffac-4ebf-8c59-1e496d6905e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:14.738451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:14.760180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 984/1969] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:14.768865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb56dc-85d9-48bb-9fc5-12847d31a281 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:14.769014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb56dc-85d9-48bb-9fc5-12847d31a281 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:14.769288 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb56dc-85d9-48bb-9fc5-12847d31a281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:14.802788 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-2fcb56dc-85d9-48bb-9fc5-12847d31a281 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:14.804183 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-2fcb56dc-85d9-48bb-9fc5-12847d31a281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:14.820498 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360891a-ffac-4ebf-8c59-1e496d6905e6 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:14.827949 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 986/1970] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:14.834158 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f360891a-ffac-4ebf-8c59-1e496d6905e6 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:14.840527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288f799-a7ce-4b1f-b7cf-06b5c5d2fdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:14.841404 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288f799-a7ce-4b1f-b7cf-06b5c5d2fdd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:14.841820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288f799-a7ce-4b1f-b7cf-06b5c5d2fdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:14.842223 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288f799-a7ce-4b1f-b7cf-06b5c5d2fdd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:14.843391 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288f799-a7ce-4b1f-b7cf-06b5c5d2fdd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:27:14.843828 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288f799-a7ce-4b1f-b7cf-06b5c5d2fdd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:14.847739 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8288f799-a7ce-4b1f-b7cf-06b5c5d2fdd2 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:14.888857 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb56dc-85d9-48bb-9fc5-12847d31a281 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:14.900266 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 985/1971] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:27:14.903430 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb56dc-85d9-48bb-9fc5-12847d31a281 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:14.908954 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78719e-329d-4122-86c7-25f9f0b75db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:14.909867 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78719e-329d-4122-86c7-25f9f0b75db1 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:14.910362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78719e-329d-4122-86c7-25f9f0b75db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:14.910828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b78719e-329d-4122-86c7-25f9f0b75db1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:14.912924 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78719e-329d-4122-86c7-25f9f0b75db1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:14.916530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b78719e-329d-4122-86c7-25f9f0b75db1 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:14.968030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 987/1972] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:27:14 2022] POST /identity/v3/projects => generated 391 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:27:14.971013 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288f799-a7ce-4b1f-b7cf-06b5c5d2fdd2 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:14.982814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8288f799-a7ce-4b1f-b7cf-06b5c5d2fdd2 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:15.058285 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a78ec4-78f8-4163-972e-6590f373cb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:15.058934 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a78ec4-78f8-4163-972e-6590f373cb21 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:15.059203 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a78ec4-78f8-4163-972e-6590f373cb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:15.059472 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a78ec4-78f8-4163-972e-6590f373cb21 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:15.060140 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a78ec4-78f8-4163-972e-6590f373cb21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:15.062099 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a78ec4-78f8-4163-972e-6590f373cb21 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:15.092202 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 986/1973] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:27:14 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:15.093962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78719e-329d-4122-86c7-25f9f0b75db1 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:15.104918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b78719e-329d-4122-86c7-25f9f0b75db1 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:15.109831 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5604e2-0ed5-4bf7-a436-3eed4906912e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:15.110651 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5604e2-0ed5-4bf7-a436-3eed4906912e admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:15.110931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5604e2-0ed5-4bf7-a436-3eed4906912e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:15.111219 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5604e2-0ed5-4bf7-a436-3eed4906912e admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:15.111880 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5604e2-0ed5-4bf7-a436-3eed4906912e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:15.113708 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5604e2-0ed5-4bf7-a436-3eed4906912e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:15.128527 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 988/1974] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:15.129045 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a78ec4-78f8-4163-972e-6590f373cb21 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:15.143955 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a78ec4-78f8-4163-972e-6590f373cb21 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:15.150144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184823b5-fc08-4447-8af2-f9da57b568ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:15.150781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184823b5-fc08-4447-8af2-f9da57b568ec admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:15.151062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184823b5-fc08-4447-8af2-f9da57b568ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:15.151334 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184823b5-fc08-4447-8af2-f9da57b568ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:15.152077 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184823b5-fc08-4447-8af2-f9da57b568ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:15.154332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184823b5-fc08-4447-8af2-f9da57b568ec admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:15.169583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 987/1975] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:15.173787 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5604e2-0ed5-4bf7-a436-3eed4906912e admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:15.190564 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5604e2-0ed5-4bf7-a436-3eed4906912e admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:15.197065 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9592cd-41c8-4e2d-839f-99f142ff2cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:15.198567 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9592cd-41c8-4e2d-839f-99f142ff2cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:15.199058 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9592cd-41c8-4e2d-839f-99f142ff2cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:15.208326 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9592cd-41c8-4e2d-839f-99f142ff2cf4 admin admin] PATH_INFO: `/v3/projects/0c0bdddb2acf431a9ae4bfcb46b719db/users/4576ac0eb9564ecdb401fc7f8f4cdc9a/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:15.238039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9592cd-41c8-4e2d-839f-99f142ff2cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c0bdddb2acf431a9ae4bfcb46b719db, user_id=4576ac0eb9564ecdb401fc7f8f4cdc9a, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:15.241016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9592cd-41c8-4e2d-839f-99f142ff2cf4 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:15.257728 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 989/1976] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:15 2022] PUT /identity/v3/projects/0c0bdddb2acf431a9ae4bfcb46b719db/users/4576ac0eb9564ecdb401fc7f8f4cdc9a/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:15.269897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29568d32-db6b-4148-bf3c-c65b8b5ac027 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:15.271146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29568d32-db6b-4148-bf3c-c65b8b5ac027 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:15.271526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29568d32-db6b-4148-bf3c-c65b8b5ac027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:15.314488 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-29568d32-db6b-4148-bf3c-c65b8b5ac027 None None] MFA Rules not processed for user `4576ac0eb9564ecdb401fc7f8f4cdc9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:15.316043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-29568d32-db6b-4148-bf3c-c65b8b5ac027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:15.415886 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 988/1977] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:15.428019 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9592cd-41c8-4e2d-839f-99f142ff2cf4 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:15.430101 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-2e9592cd-41c8-4e2d-839f-99f142ff2cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:15.515141 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9592cd-41c8-4e2d-839f-99f142ff2cf4 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:15.519172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8e8c7-5ee9-468a-8fc1-b7fbcf88f2d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:15.520603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8e8c7-5ee9-468a-8fc1-b7fbcf88f2d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:15.520603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8e8c7-5ee9-468a-8fc1-b7fbcf88f2d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:15.520603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8e8c7-5ee9-468a-8fc1-b7fbcf88f2d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:15.531406 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8e8c7-5ee9-468a-8fc1-b7fbcf88f2d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:15.533049 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8e8c7-5ee9-468a-8fc1-b7fbcf88f2d3 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:15.544364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 990/1978] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:15 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:25.412448 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=73,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:25.539258 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:35.239428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29568d32-db6b-4148-bf3c-c65b8b5ac027 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:35.241606 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-29568d32-db6b-4148-bf3c-c65b8b5ac027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:35.361958 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29568d32-db6b-4148-bf3c-c65b8b5ac027 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:35.367640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f841dc2-16e5-4932-a9b5-0d36fb505365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:35.368684 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f841dc2-16e5-4932-a9b5-0d36fb505365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:35.369128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f841dc2-16e5-4932-a9b5-0d36fb505365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:35.369553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f841dc2-16e5-4932-a9b5-0d36fb505365 admin admin] PATH_INFO: `/v3/users/c343ea204aae49fd96bce06bebd26ad7` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:35.370936 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f841dc2-16e5-4932-a9b5-0d36fb505365 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c343ea204aae49fd96bce06bebd26ad7)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:35.373232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f841dc2-16e5-4932-a9b5-0d36fb505365 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:35.473629 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4f841dc2-16e5-4932-a9b5-0d36fb505365 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c343ea204aae49fd96bce06bebd26ad7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:35.495364 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4f841dc2-16e5-4932-a9b5-0d36fb505365 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c343ea204aae49fd96bce06bebd26ad7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:35.517125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4f841dc2-16e5-4932-a9b5-0d36fb505365 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c343ea204aae49fd96bce06bebd26ad7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:35.540047 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4f841dc2-16e5-4932-a9b5-0d36fb505365 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c343ea204aae49fd96bce06bebd26ad7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:35.551658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4f841dc2-16e5-4932-a9b5-0d36fb505365 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c343ea204aae49fd96bce06bebd26ad7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:35.559005 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 989/1979] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:27:35 2022] DELETE /identity/v3/users/c343ea204aae49fd96bce06bebd26ad7 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:35.564273 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0270bf7-99b9-4f91-9c46-0f039f1c351a None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:35.564856 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0270bf7-99b9-4f91-9c46-0f039f1c351a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:35.564856 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0270bf7-99b9-4f91-9c46-0f039f1c351a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:35.588658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-b0270bf7-99b9-4f91-9c46-0f039f1c351a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:35.590351 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b0270bf7-99b9-4f91-9c46-0f039f1c351a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:35.670628 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 991/1980] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:35.681505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f841dc2-16e5-4932-a9b5-0d36fb505365 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:35.683082 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4f841dc2-16e5-4932-a9b5-0d36fb505365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:35.757118 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f841dc2-16e5-4932-a9b5-0d36fb505365 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:35.760676 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cb662-c8e9-4973-b8c7-99ec040ab8cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:35.761293 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cb662-c8e9-4973-b8c7-99ec040ab8cf service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:35.761569 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cb662-c8e9-4973-b8c7-99ec040ab8cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:35.761839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cb662-c8e9-4973-b8c7-99ec040ab8cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:35.771591 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cb662-c8e9-4973-b8c7-99ec040ab8cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:35.772807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cb662-c8e9-4973-b8c7-99ec040ab8cf service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:35.782690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 990/1981] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:36.091817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0270bf7-99b9-4f91-9c46-0f039f1c351a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:36.093247 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b0270bf7-99b9-4f91-9c46-0f039f1c351a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:36.117588 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0270bf7-99b9-4f91-9c46-0f039f1c351a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:36.123520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf41cd1-ebd7-46ef-8475-13c84da84115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:36.124276 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf41cd1-ebd7-46ef-8475-13c84da84115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:36.125027 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf41cd1-ebd7-46ef-8475-13c84da84115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:36.125297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf41cd1-ebd7-46ef-8475-13c84da84115 admin admin] PATH_INFO: `/v3/projects/48e656da916248ef9adb2cbd9b2e6770` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:36.126629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf41cd1-ebd7-46ef-8475-13c84da84115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48e656da916248ef9adb2cbd9b2e6770)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:36.128624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf41cd1-ebd7-46ef-8475-13c84da84115 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:36.205721 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0bf41cd1-ebd7-46ef-8475-13c84da84115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48e656da916248ef9adb2cbd9b2e6770', 'request_id': 'req-0bf41cd1-ebd7-46ef-8475-13c84da84115'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:36.212273 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0bf41cd1-ebd7-46ef-8475-13c84da84115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48e656da916248ef9adb2cbd9b2e6770', 'request_id': 'req-0bf41cd1-ebd7-46ef-8475-13c84da84115'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:36.220663 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0bf41cd1-ebd7-46ef-8475-13c84da84115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48e656da916248ef9adb2cbd9b2e6770', 'request_id': 'req-0bf41cd1-ebd7-46ef-8475-13c84da84115'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:36.233110 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0bf41cd1-ebd7-46ef-8475-13c84da84115 admin admin] The token cache is being invalidate because project 48e656da916248ef9adb2cbd9b2e6770 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:27:36.233321 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0bf41cd1-ebd7-46ef-8475-13c84da84115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:36.233974 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0bf41cd1-ebd7-46ef-8475-13c84da84115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:36.235622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 992/1982] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:27:36 2022] DELETE /identity/v3/projects/48e656da916248ef9adb2cbd9b2e6770 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:36.239852 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cb662-c8e9-4973-b8c7-99ec040ab8cf service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:36.250317 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cb662-c8e9-4973-b8c7-99ec040ab8cf service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:36.268759 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:36.315130 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b02328-2e99-4b71-b7de-b48abe46f483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:36.315806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b02328-2e99-4b71-b7de-b48abe46f483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:36.316136 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b02328-2e99-4b71-b7de-b48abe46f483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:36.316413 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b02328-2e99-4b71-b7de-b48abe46f483 admin admin] PATH_INFO: `/v3/users/4ea42599832f4f34bed14620e5a5aaf5` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:36.317616 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b02328-2e99-4b71-b7de-b48abe46f483 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ea42599832f4f34bed14620e5a5aaf5)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:36.319540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b02328-2e99-4b71-b7de-b48abe46f483 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:36.375258 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d2b02328-2e99-4b71-b7de-b48abe46f483 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ea42599832f4f34bed14620e5a5aaf5'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:36.383203 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d2b02328-2e99-4b71-b7de-b48abe46f483 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ea42599832f4f34bed14620e5a5aaf5'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:36.395112 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d2b02328-2e99-4b71-b7de-b48abe46f483 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ea42599832f4f34bed14620e5a5aaf5'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:36.395874 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d2b02328-2e99-4b71-b7de-b48abe46f483 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ea42599832f4f34bed14620e5a5aaf5'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:36.404077 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d2b02328-2e99-4b71-b7de-b48abe46f483 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ea42599832f4f34bed14620e5a5aaf5'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:36.410554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 991/1983] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:27:36 2022] DELETE /identity/v3/users/4ea42599832f4f34bed14620e5a5aaf5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:27:36.754355 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf41cd1-ebd7-46ef-8475-13c84da84115 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:36.757881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-0bf41cd1-ebd7-46ef-8475-13c84da84115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:36.785779 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf41cd1-ebd7-46ef-8475-13c84da84115 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:36.894319 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335fffa-c798-494a-8a42-3ca23819a831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:36.895190 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335fffa-c798-494a-8a42-3ca23819a831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:36.895423 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335fffa-c798-494a-8a42-3ca23819a831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:36.895703 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335fffa-c798-494a-8a42-3ca23819a831 admin admin] PATH_INFO: `/v3/projects/d9cd15b93e4b473eac1cfd78c3c71adb` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:36.897077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335fffa-c798-494a-8a42-3ca23819a831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9cd15b93e4b473eac1cfd78c3c71adb)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:36.910703 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335fffa-c798-494a-8a42-3ca23819a831 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:36.953156 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:36.974790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a335fffa-c798-494a-8a42-3ca23819a831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9cd15b93e4b473eac1cfd78c3c71adb', 'request_id': 'req-a335fffa-c798-494a-8a42-3ca23819a831'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:36.982053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a335fffa-c798-494a-8a42-3ca23819a831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9cd15b93e4b473eac1cfd78c3c71adb', 'request_id': 'req-a335fffa-c798-494a-8a42-3ca23819a831'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:36.989625 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a335fffa-c798-494a-8a42-3ca23819a831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9cd15b93e4b473eac1cfd78c3c71adb', 'request_id': 'req-a335fffa-c798-494a-8a42-3ca23819a831'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:37.000991 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a335fffa-c798-494a-8a42-3ca23819a831 admin admin] The token cache is being invalidate because project d9cd15b93e4b473eac1cfd78c3c71adb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:27:37.001284 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a335fffa-c798-494a-8a42-3ca23819a831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:37.001930 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a335fffa-c798-494a-8a42-3ca23819a831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:37.003935 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 993/1984] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:27:36 2022] DELETE /identity/v3/projects/d9cd15b93e4b473eac1cfd78c3c71adb => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:27:37.009182 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30505d58-5fd6-4869-91d2-12ea6003fd90 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:37.009904 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30505d58-5fd6-4869-91d2-12ea6003fd90 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:37.010175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30505d58-5fd6-4869-91d2-12ea6003fd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:37.032589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-30505d58-5fd6-4869-91d2-12ea6003fd90 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:37.033838 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-30505d58-5fd6-4869-91d2-12ea6003fd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:37.096753 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 992/1985] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:37.106038 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca1833-e126-4db4-ad0e-3912afa21ce7 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:37.106297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca1833-e126-4db4-ad0e-3912afa21ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:37.106574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eca1833-e126-4db4-ad0e-3912afa21ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:37.128811 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-5eca1833-e126-4db4-ad0e-3912afa21ce7 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:37.130046 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5eca1833-e126-4db4-ad0e-3912afa21ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:37.139433 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 994/1986] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:37.144633 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30505d58-5fd6-4869-91d2-12ea6003fd90 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:37.158625 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30505d58-5fd6-4869-91d2-12ea6003fd90 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:37.164537 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f4c97-db24-46d4-9f98-9565dbae109e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:37.165161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f4c97-db24-46d4-9f98-9565dbae109e admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:37.165432 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f4c97-db24-46d4-9f98-9565dbae109e admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:37.165704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f4c97-db24-46d4-9f98-9565dbae109e admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:37.166353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f4c97-db24-46d4-9f98-9565dbae109e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:27:37.166646 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f4c97-db24-46d4-9f98-9565dbae109e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:37.168377 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f4c97-db24-46d4-9f98-9565dbae109e admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:37.185766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 993/1987] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:27:37.187068 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca1833-e126-4db4-ad0e-3912afa21ce7 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:37.200825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eca1833-e126-4db4-ad0e-3912afa21ce7 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:37.208832 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53730b1c-b698-477e-8f9d-23342f085a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:37.210061 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53730b1c-b698-477e-8f9d-23342f085a48 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:37.210437 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53730b1c-b698-477e-8f9d-23342f085a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:37.210882 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53730b1c-b698-477e-8f9d-23342f085a48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:37.211702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53730b1c-b698-477e-8f9d-23342f085a48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:37.214286 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53730b1c-b698-477e-8f9d-23342f085a48 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:37.259628 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 995/1988] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:27:37 2022] POST /identity/v3/projects => generated 363 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:37.261106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f4c97-db24-46d4-9f98-9565dbae109e admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:37.273159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f4c97-db24-46d4-9f98-9565dbae109e admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:37.314702 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:37.356231 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b68c90-0c6f-47db-b5c8-320eca5eb7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:37.356864 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b68c90-0c6f-47db-b5c8-320eca5eb7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:37.357137 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b68c90-0c6f-47db-b5c8-320eca5eb7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:37.357409 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b68c90-0c6f-47db-b5c8-320eca5eb7c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:37.358038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b68c90-0c6f-47db-b5c8-320eca5eb7c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:37.360020 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b68c90-0c6f-47db-b5c8-320eca5eb7c2 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:37.383375 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 994/1989] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:27:37 2022] POST /identity/v3/users => generated 282 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:37.389274 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53730b1c-b698-477e-8f9d-23342f085a48 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:37.408706 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53730b1c-b698-477e-8f9d-23342f085a48 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:37.416719 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4fb9d-eded-4fa4-93f6-622315aac67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:37.417287 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4fb9d-eded-4fa4-93f6-622315aac67d admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:37.417575 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4fb9d-eded-4fa4-93f6-622315aac67d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:37.417905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4fb9d-eded-4fa4-93f6-622315aac67d admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:37.418910 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec4fb9d-eded-4fa4-93f6-622315aac67d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:37.420638 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec4fb9d-eded-4fa4-93f6-622315aac67d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:37.437012 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 996/1990] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:37.438287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b68c90-0c6f-47db-b5c8-320eca5eb7c2 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:37.449179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b68c90-0c6f-47db-b5c8-320eca5eb7c2 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:37.454157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01352521-84f7-433f-be66-171a4a44d6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:37.454677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01352521-84f7-433f-be66-171a4a44d6ed admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:37.454949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01352521-84f7-433f-be66-171a4a44d6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:37.455246 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01352521-84f7-433f-be66-171a4a44d6ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:37.455917 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01352521-84f7-433f-be66-171a4a44d6ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:37.457508 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01352521-84f7-433f-be66-171a4a44d6ed admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:37.468564 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 995/1991] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:37.473315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4fb9d-eded-4fa4-93f6-622315aac67d admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:37.491861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4fb9d-eded-4fa4-93f6-622315aac67d admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:37.498840 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360fa4e-d58f-43b9-99c7-59d0a80d4bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:37.500105 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360fa4e-d58f-43b9-99c7-59d0a80d4bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:37.500382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360fa4e-d58f-43b9-99c7-59d0a80d4bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:37.501201 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3360fa4e-d58f-43b9-99c7-59d0a80d4bdf admin admin] PATH_INFO: `/v3/projects/ec1cacb37fad4e499295edc3e317ba22/users/591de50d89e144da854de57be72477a2/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:37.525048 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3360fa4e-d58f-43b9-99c7-59d0a80d4bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec1cacb37fad4e499295edc3e317ba22, user_id=591de50d89e144da854de57be72477a2, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:37.529764 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3360fa4e-d58f-43b9-99c7-59d0a80d4bdf admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:37.546799 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 997/1992] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:37 2022] PUT /identity/v3/projects/ec1cacb37fad4e499295edc3e317ba22/users/591de50d89e144da854de57be72477a2/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:37.558603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65def67-641d-41e5-a10c-f15daded497b None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:37.558982 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65def67-641d-41e5-a10c-f15daded497b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:37.559325 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65def67-641d-41e5-a10c-f15daded497b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:37.596354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-f65def67-641d-41e5-a10c-f15daded497b None None] MFA Rules not processed for user `591de50d89e144da854de57be72477a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:37.597678 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f65def67-641d-41e5-a10c-f15daded497b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:37.677039 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 996/1993] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:37.681603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360fa4e-d58f-43b9-99c7-59d0a80d4bdf admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:37.747650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3360fa4e-d58f-43b9-99c7-59d0a80d4bdf admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:37.753125 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eabe38-0407-47c1-a9cc-27d600e03277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:37.753771 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eabe38-0407-47c1-a9cc-27d600e03277 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:37.754049 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eabe38-0407-47c1-a9cc-27d600e03277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:37.754324 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eabe38-0407-47c1-a9cc-27d600e03277 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:37.754958 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eabe38-0407-47c1-a9cc-27d600e03277 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:37.756916 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eabe38-0407-47c1-a9cc-27d600e03277 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:37.787245 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 998/1994] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:27:37 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:37.790755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65def67-641d-41e5-a10c-f15daded497b None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:37.802971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65def67-641d-41e5-a10c-f15daded497b None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:37.870362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2eafb-6e7a-4e24-94a5-79e3814e9b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:37.871688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2eafb-6e7a-4e24-94a5-79e3814e9b0b admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:37.872018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2eafb-6e7a-4e24-94a5-79e3814e9b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:37.872302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd2eafb-6e7a-4e24-94a5-79e3814e9b0b admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:37.872961 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd2eafb-6e7a-4e24-94a5-79e3814e9b0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:37.874987 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd2eafb-6e7a-4e24-94a5-79e3814e9b0b admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:37.899607 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 997/1995] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:27:37 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:37.903904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eabe38-0407-47c1-a9cc-27d600e03277 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:37.915493 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eabe38-0407-47c1-a9cc-27d600e03277 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:37.934034 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c61a3-de75-4be4-a416-194200f4c733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:37.934896 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c61a3-de75-4be4-a416-194200f4c733 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:37.935184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c61a3-de75-4be4-a416-194200f4c733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:37.935453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c61a3-de75-4be4-a416-194200f4c733 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:37.936184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c61a3-de75-4be4-a416-194200f4c733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:37.937866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c61a3-de75-4be4-a416-194200f4c733 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:37.958355 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 999/1996] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:37.963303 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2eafb-6e7a-4e24-94a5-79e3814e9b0b admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:37.975049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd2eafb-6e7a-4e24-94a5-79e3814e9b0b admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:37.993380 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aebdae-e071-42fc-9011-99dffc65180e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:37.994094 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aebdae-e071-42fc-9011-99dffc65180e admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:37.994913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aebdae-e071-42fc-9011-99dffc65180e admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:37.994913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aebdae-e071-42fc-9011-99dffc65180e admin admin] PATH_INFO: `/v3/projects/02b13178599e480da0ee171fee7cb3a9/users/1d86c0a011db4d1fa8f557b060ff90ad/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:38.028126 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aebdae-e071-42fc-9011-99dffc65180e admin admin] RBAC: Authorizing `identity:create_grant(project_id=02b13178599e480da0ee171fee7cb3a9, user_id=1d86c0a011db4d1fa8f557b060ff90ad, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:38.033371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aebdae-e071-42fc-9011-99dffc65180e admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:38.058979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 998/1997] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:37 2022] PUT /identity/v3/projects/02b13178599e480da0ee171fee7cb3a9/users/1d86c0a011db4d1fa8f557b060ff90ad/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:38.064123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c61a3-de75-4be4-a416-194200f4c733 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:38.075402 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c61a3-de75-4be4-a416-194200f4c733 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:38.103936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:38.154042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569a2a7-37ab-4d2f-aa2f-d4dfd0c7e1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:38.154885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569a2a7-37ab-4d2f-aa2f-d4dfd0c7e1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:38.155137 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569a2a7-37ab-4d2f-aa2f-d4dfd0c7e1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:38.155532 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569a2a7-37ab-4d2f-aa2f-d4dfd0c7e1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:38.156369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569a2a7-37ab-4d2f-aa2f-d4dfd0c7e1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:38.158242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569a2a7-37ab-4d2f-aa2f-d4dfd0c7e1d1 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:38.168515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1000/1998] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:38.174894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aebdae-e071-42fc-9011-99dffc65180e admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:38.184564 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:38.190490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aebdae-e071-42fc-9011-99dffc65180e admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:38.198959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32aa996-3f8b-4cdc-90b6-772d553874a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:38.199646 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32aa996-3f8b-4cdc-90b6-772d553874a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:38.199925 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32aa996-3f8b-4cdc-90b6-772d553874a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:38.200231 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32aa996-3f8b-4cdc-90b6-772d553874a7 admin admin] PATH_INFO: `/v3/projects/02b13178599e480da0ee171fee7cb3a9/users/1d86c0a011db4d1fa8f557b060ff90ad/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:38.202629 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32aa996-3f8b-4cdc-90b6-772d553874a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02b13178599e480da0ee171fee7cb3a9, user_id=1d86c0a011db4d1fa8f557b060ff90ad, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:38.205778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32aa996-3f8b-4cdc-90b6-772d553874a7 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:38.218042 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 999/1999] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:38 2022] PUT /identity/v3/projects/02b13178599e480da0ee171fee7cb3a9/users/1d86c0a011db4d1fa8f557b060ff90ad/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:38.229440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b37fd-a8d7-4fec-97fe-6a01c6956f6c None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:38.229741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b37fd-a8d7-4fec-97fe-6a01c6956f6c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:38.229993 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b37fd-a8d7-4fec-97fe-6a01c6956f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:38.262597 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-1c5b37fd-a8d7-4fec-97fe-6a01c6956f6c None None] MFA Rules not processed for user `1d86c0a011db4d1fa8f557b060ff90ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:38.263924 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1c5b37fd-a8d7-4fec-97fe-6a01c6956f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:38.348760 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32aa996-3f8b-4cdc-90b6-772d553874a7 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:38.351385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f32aa996-3f8b-4cdc-90b6-772d553874a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:38.352323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1001/2000] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:38.368279 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b37fd-a8d7-4fec-97fe-6a01c6956f6c None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:38.369639 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1c5b37fd-a8d7-4fec-97fe-6a01c6956f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:38.435391 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32aa996-3f8b-4cdc-90b6-772d553874a7 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:38.440919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2140e9-cd85-4856-bde7-823daf07b6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:38.441571 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2140e9-cd85-4856-bde7-823daf07b6df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:38.442057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2140e9-cd85-4856-bde7-823daf07b6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:38.442486 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2140e9-cd85-4856-bde7-823daf07b6df admin admin] PATH_INFO: `/v3/users/4576ac0eb9564ecdb401fc7f8f4cdc9a` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:38.443892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2140e9-cd85-4856-bde7-823daf07b6df admin admin] RBAC: Authorizing `identity:delete_user(user_id=4576ac0eb9564ecdb401fc7f8f4cdc9a)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:38.446031 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2140e9-cd85-4856-bde7-823daf07b6df admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:38.451935 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b37fd-a8d7-4fec-97fe-6a01c6956f6c None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:38.457539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745c498-39f9-42b4-9198-190b814d9fb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:38.458214 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745c498-39f9-42b4-9198-190b814d9fb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:38.458491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745c498-39f9-42b4-9198-190b814d9fb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:38.458760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745c498-39f9-42b4-9198-190b814d9fb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:38.470265 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745c498-39f9-42b4-9198-190b814d9fb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:38.471732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e745c498-39f9-42b4-9198-190b814d9fb5 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:38.483080 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1002/2001] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:38.510750 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-ef2140e9-cd85-4856-bde7-823daf07b6df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4576ac0eb9564ecdb401fc7f8f4cdc9a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:38.518392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-ef2140e9-cd85-4856-bde7-823daf07b6df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4576ac0eb9564ecdb401fc7f8f4cdc9a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:38.534538 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-ef2140e9-cd85-4856-bde7-823daf07b6df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4576ac0eb9564ecdb401fc7f8f4cdc9a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:38.535737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-ef2140e9-cd85-4856-bde7-823daf07b6df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4576ac0eb9564ecdb401fc7f8f4cdc9a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:38.544441 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-ef2140e9-cd85-4856-bde7-823daf07b6df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4576ac0eb9564ecdb401fc7f8f4cdc9a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:38.553062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1000/2002] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:27:38 2022] DELETE /identity/v3/users/4576ac0eb9564ecdb401fc7f8f4cdc9a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:27:38.557654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c665c19-2125-4bf6-8d00-b23c46c7b850 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:38.558330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c665c19-2125-4bf6-8d00-b23c46c7b850 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:38.558612 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c665c19-2125-4bf6-8d00-b23c46c7b850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:38.579248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-9c665c19-2125-4bf6-8d00-b23c46c7b850 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:38.580547 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9c665c19-2125-4bf6-8d00-b23c46c7b850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:38.598290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2140e9-cd85-4856-bde7-823daf07b6df admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:38.599778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-ef2140e9-cd85-4856-bde7-823daf07b6df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:38.632143 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:38.650443 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1003/2003] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:38.661283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c665c19-2125-4bf6-8d00-b23c46c7b850 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:38.663348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9c665c19-2125-4bf6-8d00-b23c46c7b850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:38.680180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2140e9-cd85-4856-bde7-823daf07b6df admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:38.684229 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5b76e-96a6-4319-8299-302af8e2aaa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:38.684980 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5b76e-96a6-4319-8299-302af8e2aaa6 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:38.685497 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5b76e-96a6-4319-8299-302af8e2aaa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:38.685861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5b76e-96a6-4319-8299-302af8e2aaa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:38.687766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a5e5b76e-96a6-4319-8299-302af8e2aaa6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:38.719068 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:38.758738 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c665c19-2125-4bf6-8d00-b23c46c7b850 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:38.764380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79059e-174e-44a9-b834-f2a5de01ba74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:38.765472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79059e-174e-44a9-b834-f2a5de01ba74 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:38.767294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79059e-174e-44a9-b834-f2a5de01ba74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:38.768010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79059e-174e-44a9-b834-f2a5de01ba74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:38.787892 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79059e-174e-44a9-b834-f2a5de01ba74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:38.789476 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79059e-174e-44a9-b834-f2a5de01ba74 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:38.797272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5b76e-96a6-4319-8299-302af8e2aaa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:38.799333 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5b76e-96a6-4319-8299-302af8e2aaa6 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:38.800863 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1004/2004] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:38.814929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1001/2005] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:27:39.214866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79059e-174e-44a9-b834-f2a5de01ba74 service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:39.216644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-3c79059e-174e-44a9-b834-f2a5de01ba74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:39.232591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79059e-174e-44a9-b834-f2a5de01ba74 service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:39.248240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2174d-2835-4e02-910b-2ca779351130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:39.249400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2174d-2835-4e02-910b-2ca779351130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:39.250406 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2174d-2835-4e02-910b-2ca779351130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:39.250933 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2174d-2835-4e02-910b-2ca779351130 admin admin] PATH_INFO: `/v3/projects/0c0bdddb2acf431a9ae4bfcb46b719db` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:39.253216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2174d-2835-4e02-910b-2ca779351130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c0bdddb2acf431a9ae4bfcb46b719db)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:39.267264 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2174d-2835-4e02-910b-2ca779351130 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:39.325700 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-97e2174d-2835-4e02-910b-2ca779351130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c0bdddb2acf431a9ae4bfcb46b719db', 'request_id': 'req-97e2174d-2835-4e02-910b-2ca779351130'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:39.342001 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-97e2174d-2835-4e02-910b-2ca779351130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c0bdddb2acf431a9ae4bfcb46b719db', 'request_id': 'req-97e2174d-2835-4e02-910b-2ca779351130'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:39.349427 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-97e2174d-2835-4e02-910b-2ca779351130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c0bdddb2acf431a9ae4bfcb46b719db', 'request_id': 'req-97e2174d-2835-4e02-910b-2ca779351130'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:39.359953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-97e2174d-2835-4e02-910b-2ca779351130 admin admin] The token cache is being invalidate because project 0c0bdddb2acf431a9ae4bfcb46b719db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:27:39.360363 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-97e2174d-2835-4e02-910b-2ca779351130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:39.361096 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-97e2174d-2835-4e02-910b-2ca779351130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:39.362602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1005/2006] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:27:39 2022] DELETE /identity/v3/projects/0c0bdddb2acf431a9ae4bfcb46b719db => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:39.367729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a259ee-c1e0-4cb4-bb28-c31c8608cb99 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:39.368241 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a259ee-c1e0-4cb4-bb28-c31c8608cb99 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:39.368356 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a259ee-c1e0-4cb4-bb28-c31c8608cb99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:39.389785 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-59a259ee-c1e0-4cb4-bb28-c31c8608cb99 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:39.391007 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-59a259ee-c1e0-4cb4-bb28-c31c8608cb99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:39.453095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1002/2007] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:39.461297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764975f0-fa30-4526-b7fb-599c95c8bda0 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:39.461578 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764975f0-fa30-4526-b7fb-599c95c8bda0 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:39.461855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764975f0-fa30-4526-b7fb-599c95c8bda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:39.482287 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-764975f0-fa30-4526-b7fb-599c95c8bda0 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:39.483517 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-764975f0-fa30-4526-b7fb-599c95c8bda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:39.495406 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1006/2008] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:39.496691 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a259ee-c1e0-4cb4-bb28-c31c8608cb99 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:39.507303 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a259ee-c1e0-4cb4-bb28-c31c8608cb99 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:39.512525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8b366f-7d22-4fc8-ba79-fe1917aa134a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:39.513165 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b366f-7d22-4fc8-ba79-fe1917aa134a admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:39.513432 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b366f-7d22-4fc8-ba79-fe1917aa134a admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:39.513700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8b366f-7d22-4fc8-ba79-fe1917aa134a admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:39.514514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8b366f-7d22-4fc8-ba79-fe1917aa134a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:27:39.514820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8b366f-7d22-4fc8-ba79-fe1917aa134a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:39.516533 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8b366f-7d22-4fc8-ba79-fe1917aa134a admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:39.529018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1003/2009] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:27:39.533519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764975f0-fa30-4526-b7fb-599c95c8bda0 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:39.542789 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764975f0-fa30-4526-b7fb-599c95c8bda0 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:39.548168 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d247b-5d16-4740-ac33-3cf3534294e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:39.548791 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d247b-5d16-4740-ac33-3cf3534294e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:39.549069 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d247b-5d16-4740-ac33-3cf3534294e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:39.549358 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d247b-5d16-4740-ac33-3cf3534294e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:39.550035 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d247b-5d16-4740-ac33-3cf3534294e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:39.552077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574d247b-5d16-4740-ac33-3cf3534294e4 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:39.584029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1007/2010] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:27:39 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:39.588719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8b366f-7d22-4fc8-ba79-fe1917aa134a admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:39.599795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8b366f-7d22-4fc8-ba79-fe1917aa134a admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:39.662660 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7c1fe-e9f1-4b16-bdb5-fe3aba942a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:39.663504 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7c1fe-e9f1-4b16-bdb5-fe3aba942a59 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:39.663782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7c1fe-e9f1-4b16-bdb5-fe3aba942a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:39.664113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7c1fe-e9f1-4b16-bdb5-fe3aba942a59 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:39.664763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7c1fe-e9f1-4b16-bdb5-fe3aba942a59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:39.666874 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7c1fe-e9f1-4b16-bdb5-fe3aba942a59 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:39.695880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1004/2011] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:27:39 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:27:39.701348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d247b-5d16-4740-ac33-3cf3534294e4 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:39.716397 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d247b-5d16-4740-ac33-3cf3534294e4 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:39.723222 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8854d1-99dd-4fe5-8cbc-b5463880f5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:39.724157 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8854d1-99dd-4fe5-8cbc-b5463880f5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:39.724641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8854d1-99dd-4fe5-8cbc-b5463880f5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:39.725435 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8854d1-99dd-4fe5-8cbc-b5463880f5c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:39.726538 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8854d1-99dd-4fe5-8cbc-b5463880f5c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:39.728744 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8854d1-99dd-4fe5-8cbc-b5463880f5c4 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:39.746784 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1008/2012] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:27:39.748884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7c1fe-e9f1-4b16-bdb5-fe3aba942a59 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:39.760084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7c1fe-e9f1-4b16-bdb5-fe3aba942a59 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:39.764968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef7b6a8-bcc3-4491-a3ec-e4a427b126ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:39.765659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7b6a8-bcc3-4491-a3ec-e4a427b126ab admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:39.765943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7b6a8-bcc3-4491-a3ec-e4a427b126ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:39.766223 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef7b6a8-bcc3-4491-a3ec-e4a427b126ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:39.766975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7b6a8-bcc3-4491-a3ec-e4a427b126ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:39.768771 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef7b6a8-bcc3-4491-a3ec-e4a427b126ab admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:39.786599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1005/2013] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:39.790295 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8854d1-99dd-4fe5-8cbc-b5463880f5c4 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:39.801562 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8854d1-99dd-4fe5-8cbc-b5463880f5c4 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:39.806689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1e5c4-c02f-42fa-b1ba-2d9bd117e132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:39.807377 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1e5c4-c02f-42fa-b1ba-2d9bd117e132 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:39.807636 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1e5c4-c02f-42fa-b1ba-2d9bd117e132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:39.807941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1e5c4-c02f-42fa-b1ba-2d9bd117e132 admin admin] PATH_INFO: `/v3/projects/6e5e8fba8d65419393ba9829cafdd5b9/users/dbec912e4d5c4c6aabed5a98b6aa632d/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:39.828950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1e5c4-c02f-42fa-b1ba-2d9bd117e132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e5e8fba8d65419393ba9829cafdd5b9, user_id=dbec912e4d5c4c6aabed5a98b6aa632d, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:39.832364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1e5c4-c02f-42fa-b1ba-2d9bd117e132 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:39.850390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1009/2014] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:39 2022] PUT /identity/v3/projects/6e5e8fba8d65419393ba9829cafdd5b9/users/dbec912e4d5c4c6aabed5a98b6aa632d/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:39.855453 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518df841-04e4-4dd3-9b00-d2f48174a2d1 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:39.856574 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518df841-04e4-4dd3-9b00-d2f48174a2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:39.857184 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518df841-04e4-4dd3-9b00-d2f48174a2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:39.877556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-518df841-04e4-4dd3-9b00-d2f48174a2d1 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:39.879342 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-518df841-04e4-4dd3-9b00-d2f48174a2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:39.905055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:39.948962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1006/2015] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:39.964579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1e5c4-c02f-42fa-b1ba-2d9bd117e132 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:39.966933 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-85f1e5c4-c02f-42fa-b1ba-2d9bd117e132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:39.982474 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:40.051850 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1e5c4-c02f-42fa-b1ba-2d9bd117e132 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:40.064770 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce9a72-783b-49c4-b5b6-8d491635cdba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:40.064770 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce9a72-783b-49c4-b5b6-8d491635cdba service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:40.064770 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce9a72-783b-49c4-b5b6-8d491635cdba service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:40.064770 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce9a72-783b-49c4-b5b6-8d491635cdba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:40.073273 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce9a72-783b-49c4-b5b6-8d491635cdba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:40.074861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce9a72-783b-49c4-b5b6-8d491635cdba service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:40.087226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1010/2016] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:42.189161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518df841-04e4-4dd3-9b00-d2f48174a2d1 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:42.191297 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-518df841-04e4-4dd3-9b00-d2f48174a2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:42.208257 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518df841-04e4-4dd3-9b00-d2f48174a2d1 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:42.213037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980bfb5-9599-4310-ae80-1b27786d5caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:42.213629 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980bfb5-9599-4310-ae80-1b27786d5caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:42.213898 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980bfb5-9599-4310-ae80-1b27786d5caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:42.214170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980bfb5-9599-4310-ae80-1b27786d5caf admin admin] PATH_INFO: `/v3/users/a02a5e8b67c94800a6f0f3604366f80a` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:42.215298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980bfb5-9599-4310-ae80-1b27786d5caf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a02a5e8b67c94800a6f0f3604366f80a)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:42.217308 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980bfb5-9599-4310-ae80-1b27786d5caf admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:42.279620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d980bfb5-9599-4310-ae80-1b27786d5caf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a02a5e8b67c94800a6f0f3604366f80a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:42.287351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d980bfb5-9599-4310-ae80-1b27786d5caf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a02a5e8b67c94800a6f0f3604366f80a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:42.302765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d980bfb5-9599-4310-ae80-1b27786d5caf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a02a5e8b67c94800a6f0f3604366f80a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:42.303868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d980bfb5-9599-4310-ae80-1b27786d5caf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a02a5e8b67c94800a6f0f3604366f80a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:42.316070 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d980bfb5-9599-4310-ae80-1b27786d5caf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a02a5e8b67c94800a6f0f3604366f80a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:42.327159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1007/2017] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:27:42 2022] DELETE /identity/v3/users/a02a5e8b67c94800a6f0f3604366f80a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:43.431961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce9a72-783b-49c4-b5b6-8d491635cdba service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:43.445137 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce9a72-783b-49c4-b5b6-8d491635cdba service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:43.530684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224be69-2fc2-47da-ab30-bcb50a27d69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:43.531412 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224be69-2fc2-47da-ab30-bcb50a27d69b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:43.532198 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224be69-2fc2-47da-ab30-bcb50a27d69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:43.532817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224be69-2fc2-47da-ab30-bcb50a27d69b admin admin] PATH_INFO: `/v3/projects/8df3773770a54a629538b2cf6d70b8d4` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:43.534628 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224be69-2fc2-47da-ab30-bcb50a27d69b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8df3773770a54a629538b2cf6d70b8d4)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:43.538943 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224be69-2fc2-47da-ab30-bcb50a27d69b admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:43.626225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5224be69-2fc2-47da-ab30-bcb50a27d69b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8df3773770a54a629538b2cf6d70b8d4', 'request_id': 'req-5224be69-2fc2-47da-ab30-bcb50a27d69b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:43.669163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5224be69-2fc2-47da-ab30-bcb50a27d69b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8df3773770a54a629538b2cf6d70b8d4', 'request_id': 'req-5224be69-2fc2-47da-ab30-bcb50a27d69b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:43.686845 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5224be69-2fc2-47da-ab30-bcb50a27d69b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8df3773770a54a629538b2cf6d70b8d4', 'request_id': 'req-5224be69-2fc2-47da-ab30-bcb50a27d69b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:43.715519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5224be69-2fc2-47da-ab30-bcb50a27d69b admin admin] The token cache is being invalidate because project 8df3773770a54a629538b2cf6d70b8d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:27:43.715519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5224be69-2fc2-47da-ab30-bcb50a27d69b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:43.715519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5224be69-2fc2-47da-ab30-bcb50a27d69b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:43.715519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1011/2018] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:27:43 2022] DELETE /identity/v3/projects/8df3773770a54a629538b2cf6d70b8d4 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:43.721269 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980bfb5-9599-4310-ae80-1b27786d5caf admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:43.725777 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-d980bfb5-9599-4310-ae80-1b27786d5caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:43.756216 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980bfb5-9599-4310-ae80-1b27786d5caf admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:43.854451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23cd8b7-961e-4e19-b0a6-7ed6dd97c59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:43.855397 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cd8b7-961e-4e19-b0a6-7ed6dd97c59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:43.855699 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cd8b7-961e-4e19-b0a6-7ed6dd97c59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:43.856307 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23cd8b7-961e-4e19-b0a6-7ed6dd97c59b admin admin] PATH_INFO: `/v3/users/02182da4694b4a1aa2b6a0b8b1013971` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:43.857539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23cd8b7-961e-4e19-b0a6-7ed6dd97c59b admin admin] RBAC: Authorizing `identity:delete_user(user_id=02182da4694b4a1aa2b6a0b8b1013971)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:43.859513 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23cd8b7-961e-4e19-b0a6-7ed6dd97c59b admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:43.947421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:43.951151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d23cd8b7-961e-4e19-b0a6-7ed6dd97c59b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02182da4694b4a1aa2b6a0b8b1013971'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:43.965097 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d23cd8b7-961e-4e19-b0a6-7ed6dd97c59b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02182da4694b4a1aa2b6a0b8b1013971'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:43.979900 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d23cd8b7-961e-4e19-b0a6-7ed6dd97c59b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02182da4694b4a1aa2b6a0b8b1013971'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:43.980781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d23cd8b7-961e-4e19-b0a6-7ed6dd97c59b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02182da4694b4a1aa2b6a0b8b1013971'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:43.993194 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d23cd8b7-961e-4e19-b0a6-7ed6dd97c59b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02182da4694b4a1aa2b6a0b8b1013971'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:44.003719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1008/2019] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:27:43 2022] DELETE /identity/v3/users/02182da4694b4a1aa2b6a0b8b1013971 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:44.270133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224be69-2fc2-47da-ab30-bcb50a27d69b admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:44.271535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5224be69-2fc2-47da-ab30-bcb50a27d69b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:44.295762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224be69-2fc2-47da-ab30-bcb50a27d69b admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:44.399346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:44.438826 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79a4aa-24d5-4392-8a87-8ea5df49661c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:44.439770 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79a4aa-24d5-4392-8a87-8ea5df49661c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:44.440084 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79a4aa-24d5-4392-8a87-8ea5df49661c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:44.440622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79a4aa-24d5-4392-8a87-8ea5df49661c admin admin] PATH_INFO: `/v3/projects/210f612935e743f3a41d0c1a2bb986ca` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:44.443409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79a4aa-24d5-4392-8a87-8ea5df49661c admin admin] RBAC: Authorizing `identity:delete_project(project_id=210f612935e743f3a41d0c1a2bb986ca)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:44.448010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79a4aa-24d5-4392-8a87-8ea5df49661c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:44.576567 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1e79a4aa-24d5-4392-8a87-8ea5df49661c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '210f612935e743f3a41d0c1a2bb986ca', 'request_id': 'req-1e79a4aa-24d5-4392-8a87-8ea5df49661c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:44.587025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1e79a4aa-24d5-4392-8a87-8ea5df49661c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '210f612935e743f3a41d0c1a2bb986ca', 'request_id': 'req-1e79a4aa-24d5-4392-8a87-8ea5df49661c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:44.601283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1e79a4aa-24d5-4392-8a87-8ea5df49661c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '210f612935e743f3a41d0c1a2bb986ca', 'request_id': 'req-1e79a4aa-24d5-4392-8a87-8ea5df49661c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:44.623533 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1e79a4aa-24d5-4392-8a87-8ea5df49661c admin admin] The token cache is being invalidate because project 210f612935e743f3a41d0c1a2bb986ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:27:44.624091 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1e79a4aa-24d5-4392-8a87-8ea5df49661c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:44.625020 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1e79a4aa-24d5-4392-8a87-8ea5df49661c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:44.628574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1012/2020] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:27:44 2022] DELETE /identity/v3/projects/210f612935e743f3a41d0c1a2bb986ca => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:44.633814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742a4f6-c301-483c-a06b-bd1041797a88 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:44.634552 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742a4f6-c301-483c-a06b-bd1041797a88 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:44.635675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9742a4f6-c301-483c-a06b-bd1041797a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:44.664955 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-9742a4f6-c301-483c-a06b-bd1041797a88 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:44.667735 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9742a4f6-c301-483c-a06b-bd1041797a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:44.771089 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1009/2021] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:27:44.781728 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2c606-9706-4153-9b2c-568edc63650b None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:44.782112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2c606-9706-4153-9b2c-568edc63650b None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:44.782362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2c606-9706-4153-9b2c-568edc63650b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:44.840269 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-93a2c606-9706-4153-9b2c-568edc63650b None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:44.841513 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-93a2c606-9706-4153-9b2c-568edc63650b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:44.882324 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1013/2022] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:44.882649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742a4f6-c301-483c-a06b-bd1041797a88 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:44.894587 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9742a4f6-c301-483c-a06b-bd1041797a88 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:44.900951 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbac33d-843f-4616-950e-d4191fde7b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:44.901726 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbac33d-843f-4616-950e-d4191fde7b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:44.903165 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbac33d-843f-4616-950e-d4191fde7b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:44.904319 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbac33d-843f-4616-950e-d4191fde7b6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:44.904319 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbac33d-843f-4616-950e-d4191fde7b6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:27:44.904791 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbac33d-843f-4616-950e-d4191fde7b6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:44.906776 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbac33d-843f-4616-950e-d4191fde7b6d admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:44.946096 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1010/2023] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:27:44.947277 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2c606-9706-4153-9b2c-568edc63650b None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:44.968417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a2c606-9706-4153-9b2c-568edc63650b None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:44.968417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254f60d-e072-45e3-8288-220505d08da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:44.968417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254f60d-e072-45e3-8288-220505d08da1 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:44.968417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254f60d-e072-45e3-8288-220505d08da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:44.968706 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254f60d-e072-45e3-8288-220505d08da1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:44.968706 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8254f60d-e072-45e3-8288-220505d08da1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:44.968830 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8254f60d-e072-45e3-8288-220505d08da1 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:45.018624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1014/2024] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:27:44 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:45.019933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbac33d-843f-4616-950e-d4191fde7b6d admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:45.034771 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbac33d-843f-4616-950e-d4191fde7b6d admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:45.140457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254f60d-e072-45e3-8288-220505d08da1 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:45.142868 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8254f60d-e072-45e3-8288-220505d08da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:45.205828 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f14116-b96f-48b2-9851-5ec2443f12dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:45.206484 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f14116-b96f-48b2-9851-5ec2443f12dc admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:45.206758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f14116-b96f-48b2-9851-5ec2443f12dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:45.207025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f14116-b96f-48b2-9851-5ec2443f12dc admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:45.207671 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f14116-b96f-48b2-9851-5ec2443f12dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:45.210704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f14116-b96f-48b2-9851-5ec2443f12dc admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:45.258908 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8254f60d-e072-45e3-8288-220505d08da1 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:45.259287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1011/2025] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:27:45 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:45.265734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f14116-b96f-48b2-9851-5ec2443f12dc admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:45.292057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f14116-b96f-48b2-9851-5ec2443f12dc admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:45.308475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda56c6-5e24-44ec-890a-575c44182cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:45.310058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0217e-2cc2-4912-ac23-5f4a3dc78042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:45.310761 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0217e-2cc2-4912-ac23-5f4a3dc78042 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:45.311029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0217e-2cc2-4912-ac23-5f4a3dc78042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:45.311291 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0217e-2cc2-4912-ac23-5f4a3dc78042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:45.311767 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda56c6-5e24-44ec-890a-575c44182cfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:45.311959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0217e-2cc2-4912-ac23-5f4a3dc78042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:45.314094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda56c6-5e24-44ec-890a-575c44182cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:45.314094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda56c6-5e24-44ec-890a-575c44182cfb admin admin] PATH_INFO: `/v3/users/5ce40a75512a4ff9971552ffc54136c3` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:45.319419 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda56c6-5e24-44ec-890a-575c44182cfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce40a75512a4ff9971552ffc54136c3)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:45.326299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0217e-2cc2-4912-ac23-5f4a3dc78042 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:45.329530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda56c6-5e24-44ec-890a-575c44182cfb admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:45.356538 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1012/2026] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:45.361080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0217e-2cc2-4912-ac23-5f4a3dc78042 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:45.390956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0217e-2cc2-4912-ac23-5f4a3dc78042 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:45.415611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765d9e2-69b5-4fde-96ad-36315d9d1a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:45.416879 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765d9e2-69b5-4fde-96ad-36315d9d1a03 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:45.417835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765d9e2-69b5-4fde-96ad-36315d9d1a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:45.418365 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765d9e2-69b5-4fde-96ad-36315d9d1a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:45.420150 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765d9e2-69b5-4fde-96ad-36315d9d1a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:45.422420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765d9e2-69b5-4fde-96ad-36315d9d1a03 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:45.452575 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-3bda56c6-5e24-44ec-890a-575c44182cfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce40a75512a4ff9971552ffc54136c3'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:45.454967 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:45.465435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1013/2027] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:27:45.470101 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765d9e2-69b5-4fde-96ad-36315d9d1a03 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:45.488660 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765d9e2-69b5-4fde-96ad-36315d9d1a03 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:45.489617 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-3bda56c6-5e24-44ec-890a-575c44182cfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce40a75512a4ff9971552ffc54136c3'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:45.498328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567e741-d51a-4708-aa92-eb2891b1280a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:45.499210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567e741-d51a-4708-aa92-eb2891b1280a admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:45.499549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567e741-d51a-4708-aa92-eb2891b1280a admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:45.500017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567e741-d51a-4708-aa92-eb2891b1280a admin admin] PATH_INFO: `/v3/projects/349f204957e84d5f89222b84717c077a/users/ed5522645027406abc9741967cd22501/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:45.504446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-3bda56c6-5e24-44ec-890a-575c44182cfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce40a75512a4ff9971552ffc54136c3'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:45.505411 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-3bda56c6-5e24-44ec-890a-575c44182cfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce40a75512a4ff9971552ffc54136c3'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:45.506232 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-3bda56c6-5e24-44ec-890a-575c44182cfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce40a75512a4ff9971552ffc54136c3'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:45.524974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:45.530560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567e741-d51a-4708-aa92-eb2891b1280a admin admin] RBAC: Authorizing `identity:create_grant(project_id=349f204957e84d5f89222b84717c077a, user_id=ed5522645027406abc9741967cd22501, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:45.533638 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1015/2028] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:27:45 2022] DELETE /identity/v3/users/5ce40a75512a4ff9971552ffc54136c3 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:27:45.535275 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567e741-d51a-4708-aa92-eb2891b1280a admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:45.562726 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1014/2029] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:45 2022] PUT /identity/v3/projects/349f204957e84d5f89222b84717c077a/users/ed5522645027406abc9741967cd22501/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:27:45.572879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561a193-75d7-4d11-b781-96276f313f27 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:45.573716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561a193-75d7-4d11-b781-96276f313f27 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:45.573716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561a193-75d7-4d11-b781-96276f313f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:45.605085 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-8561a193-75d7-4d11-b781-96276f313f27 None None] MFA Rules not processed for user `ed5522645027406abc9741967cd22501`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:45.606335 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8561a193-75d7-4d11-b781-96276f313f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:45.692697 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1016/2030] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:45.697075 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567e741-d51a-4708-aa92-eb2891b1280a admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:45.699055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-5567e741-d51a-4708-aa92-eb2891b1280a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:45.834228 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567e741-d51a-4708-aa92-eb2891b1280a admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:45.843935 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241e219-09ac-4222-bc61-38d6d06f3f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:45.844855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241e219-09ac-4222-bc61-38d6d06f3f25 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:45.845289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241e219-09ac-4222-bc61-38d6d06f3f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:45.845712 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241e219-09ac-4222-bc61-38d6d06f3f25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:45.846545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241e219-09ac-4222-bc61-38d6d06f3f25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:45.848931 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241e219-09ac-4222-bc61-38d6d06f3f25 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:45.901999 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1015/2031] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:27:45 2022] POST /identity/v3/projects => generated 385 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:27:45.902228 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561a193-75d7-4d11-b781-96276f313f27 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:45.916338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561a193-75d7-4d11-b781-96276f313f27 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:45.944368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241e219-09ac-4222-bc61-38d6d06f3f25 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:45.947144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9241e219-09ac-4222-bc61-38d6d06f3f25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:46.068950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a27f78-51fc-4d14-ac8b-35c814ae1d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:46.069591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a27f78-51fc-4d14-ac8b-35c814ae1d98 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:46.069877 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a27f78-51fc-4d14-ac8b-35c814ae1d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:46.070155 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a27f78-51fc-4d14-ac8b-35c814ae1d98 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:46.070810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a27f78-51fc-4d14-ac8b-35c814ae1d98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:46.075579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a27f78-51fc-4d14-ac8b-35c814ae1d98 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:46.107085 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1017/2032] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:27:45 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:27:46.111557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a27f78-51fc-4d14-ac8b-35c814ae1d98 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:46.121090 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241e219-09ac-4222-bc61-38d6d06f3f25 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:46.122571 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a27f78-51fc-4d14-ac8b-35c814ae1d98 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:46.126559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa264583-3d87-46f2-87ca-57cecfef0317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:46.127348 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa264583-3d87-46f2-87ca-57cecfef0317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:46.127637 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa264583-3d87-46f2-87ca-57cecfef0317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:46.127911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa264583-3d87-46f2-87ca-57cecfef0317 admin admin] PATH_INFO: `/v3/projects/d300969a86954ad7ab74b743e72bcaf1` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:46.128224 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c61a5-f59f-487c-8276-65d954ef8f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:46.128970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c61a5-f59f-487c-8276-65d954ef8f43 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:46.129270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa264583-3d87-46f2-87ca-57cecfef0317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d300969a86954ad7ab74b743e72bcaf1)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:46.129397 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c61a5-f59f-487c-8276-65d954ef8f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:46.129697 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c61a5-f59f-487c-8276-65d954ef8f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:46.130564 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c61a5-f59f-487c-8276-65d954ef8f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:46.131206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa264583-3d87-46f2-87ca-57cecfef0317 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:46.133389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c61a5-f59f-487c-8276-65d954ef8f43 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:46.150492 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1018/2033] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:27:46.152275 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c61a5-f59f-487c-8276-65d954ef8f43 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:46.164461 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c61a5-f59f-487c-8276-65d954ef8f43 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:46.173298 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b694b95-8122-4981-9e23-85fa615a5da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:46.173519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b694b95-8122-4981-9e23-85fa615a5da6 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:46.173801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b694b95-8122-4981-9e23-85fa615a5da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:46.174150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b694b95-8122-4981-9e23-85fa615a5da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:46.174789 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b694b95-8122-4981-9e23-85fa615a5da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:46.176767 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b694b95-8122-4981-9e23-85fa615a5da6 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:46.193538 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1019/2034] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:46.194664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b694b95-8122-4981-9e23-85fa615a5da6 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:46.207631 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b694b95-8122-4981-9e23-85fa615a5da6 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:46.222701 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-fa264583-3d87-46f2-87ca-57cecfef0317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd300969a86954ad7ab74b743e72bcaf1', 'request_id': 'req-fa264583-3d87-46f2-87ca-57cecfef0317'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:46.232225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-fa264583-3d87-46f2-87ca-57cecfef0317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd300969a86954ad7ab74b743e72bcaf1', 'request_id': 'req-fa264583-3d87-46f2-87ca-57cecfef0317'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:46.246603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:46.249566 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-fa264583-3d87-46f2-87ca-57cecfef0317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd300969a86954ad7ab74b743e72bcaf1', 'request_id': 'req-fa264583-3d87-46f2-87ca-57cecfef0317'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:46.272337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-fa264583-3d87-46f2-87ca-57cecfef0317 admin admin] The token cache is being invalidate because project d300969a86954ad7ab74b743e72bcaf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:27:46.272637 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-fa264583-3d87-46f2-87ca-57cecfef0317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:46.273330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-fa264583-3d87-46f2-87ca-57cecfef0317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:46.276119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1016/2035] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:27:45 2022] DELETE /identity/v3/projects/d300969a86954ad7ab74b743e72bcaf1 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:46.282309 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b89434-be15-4ab0-9656-f25b62b946a4 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:46.282825 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b89434-be15-4ab0-9656-f25b62b946a4 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:46.283393 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b89434-be15-4ab0-9656-f25b62b946a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:46.311593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:46.323272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-f7b89434-be15-4ab0-9656-f25b62b946a4 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:46.324924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f7b89434-be15-4ab0-9656-f25b62b946a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:46.347148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63406a5b-0cfb-491d-b342-b5e6b5ee534a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:46.347975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63406a5b-0cfb-491d-b342-b5e6b5ee534a admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:46.348715 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63406a5b-0cfb-491d-b342-b5e6b5ee534a admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:46.349202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63406a5b-0cfb-491d-b342-b5e6b5ee534a admin admin] PATH_INFO: `/v3/projects/a247126043d847cd99ffd80841064252/users/43abbe23782e462080db18f07d2298d7/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:46.373403 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63406a5b-0cfb-491d-b342-b5e6b5ee534a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a247126043d847cd99ffd80841064252, user_id=43abbe23782e462080db18f07d2298d7, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:46.381669 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63406a5b-0cfb-491d-b342-b5e6b5ee534a admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:46.412112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1020/2036] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:46 2022] PUT /identity/v3/projects/a247126043d847cd99ffd80841064252/users/43abbe23782e462080db18f07d2298d7/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:46.432507 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba30509-f0cc-4fe0-8b44-ec279b23ce57 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:46.432785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba30509-f0cc-4fe0-8b44-ec279b23ce57 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:46.433055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba30509-f0cc-4fe0-8b44-ec279b23ce57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:46.433492 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1017/2037] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:27:46.436364 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33b6e2-2dd7-403c-9814-b9ccfc715ad1 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:46.436849 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33b6e2-2dd7-403c-9814-b9ccfc715ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:46.437318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33b6e2-2dd7-403c-9814-b9ccfc715ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:46.473419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-af33b6e2-2dd7-403c-9814-b9ccfc715ad1 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:46.475386 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-1ba30509-f0cc-4fe0-8b44-ec279b23ce57 None None] MFA Rules not processed for user `43abbe23782e462080db18f07d2298d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:46.476359 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-af33b6e2-2dd7-403c-9814-b9ccfc715ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:46.476596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1ba30509-f0cc-4fe0-8b44-ec279b23ce57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:46.602511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1021/2038] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:27:46.604131 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1018/2039] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:46.606488 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba30509-f0cc-4fe0-8b44-ec279b23ce57 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:46.607922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1ba30509-f0cc-4fe0-8b44-ec279b23ce57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:46.610065 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33b6e2-2dd7-403c-9814-b9ccfc715ad1 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:46.624014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba30509-f0cc-4fe0-8b44-ec279b23ce57 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:46.625109 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33b6e2-2dd7-403c-9814-b9ccfc715ad1 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:46.629176 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482640cf-44f1-43db-9a40-2c0af867792a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:46.629816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482640cf-44f1-43db-9a40-2c0af867792a admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:46.630084 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482640cf-44f1-43db-9a40-2c0af867792a admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:46.630354 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482640cf-44f1-43db-9a40-2c0af867792a admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:46.631027 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482640cf-44f1-43db-9a40-2c0af867792a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:46.631700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e2673-61eb-472c-80ac-932d2c4b5db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:46.632330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e2673-61eb-472c-80ac-932d2c4b5db2 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:46.632749 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e2673-61eb-472c-80ac-932d2c4b5db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:46.633032 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482640cf-44f1-43db-9a40-2c0af867792a admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:46.633195 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e2673-61eb-472c-80ac-932d2c4b5db2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:46.635649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e2673-61eb-472c-80ac-932d2c4b5db2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:27:46.636229 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e2673-61eb-472c-80ac-932d2c4b5db2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:46.641993 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0e2673-61eb-472c-80ac-932d2c4b5db2 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:46.657597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1019/2040] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:27:46.663456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e2673-61eb-472c-80ac-932d2c4b5db2 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:46.674347 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1022/2041] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:27:46 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:46.675539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482640cf-44f1-43db-9a40-2c0af867792a admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:46.677666 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e2673-61eb-472c-80ac-932d2c4b5db2 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:46.779696 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7792bca-3914-4e20-a7b2-2a795c360462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:46.780713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7792bca-3914-4e20-a7b2-2a795c360462 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:46.782416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7792bca-3914-4e20-a7b2-2a795c360462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:46.782880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7792bca-3914-4e20-a7b2-2a795c360462 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:46.783922 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7792bca-3914-4e20-a7b2-2a795c360462 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:46.786247 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7792bca-3914-4e20-a7b2-2a795c360462 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:46.823255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1020/2042] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:27:46 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:46.824598 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7792bca-3914-4e20-a7b2-2a795c360462 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:46.831046 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482640cf-44f1-43db-9a40-2c0af867792a admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:46.851726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:46.869691 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:46.911939 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54bd34-7035-4f7a-9f3e-31974c970ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:46.912593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54bd34-7035-4f7a-9f3e-31974c970ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:46.913476 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54bd34-7035-4f7a-9f3e-31974c970ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:46.913476 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54bd34-7035-4f7a-9f3e-31974c970ef4 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:46.913792 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54bd34-7035-4f7a-9f3e-31974c970ef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:46.915805 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54bd34-7035-4f7a-9f3e-31974c970ef4 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:46.920242 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7792bca-3914-4e20-a7b2-2a795c360462 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:46.934969 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11430ea2-fec0-40f5-92dc-c745c61a8ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:46.935924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11430ea2-fec0-40f5-92dc-c745c61a8ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:46.936134 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11430ea2-fec0-40f5-92dc-c745c61a8ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:46.936292 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11430ea2-fec0-40f5-92dc-c745c61a8ab3 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:46.936959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11430ea2-fec0-40f5-92dc-c745c61a8ab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:46.939319 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11430ea2-fec0-40f5-92dc-c745c61a8ab3 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:46.945359 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1023/2043] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:27:46 2022] POST /identity/v3/users => generated 299 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:46.949823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893b37c-8925-4e72-a030-719734776a1f None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:46.950071 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893b37c-8925-4e72-a030-719734776a1f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:46.950343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6893b37c-8925-4e72-a030-719734776a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:46.967690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1021/2044] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:27:46 2022] POST /identity/v3/users => generated 294 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:46.967690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11430ea2-fec0-40f5-92dc-c745c61a8ab3 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:46.977509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11430ea2-fec0-40f5-92dc-c745c61a8ab3 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:46.983100 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd190904-92e4-4af6-b8d0-6e17a143a3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:46.983961 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190904-92e4-4af6-b8d0-6e17a143a3b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:46.984421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190904-92e4-4af6-b8d0-6e17a143a3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:46.984843 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190904-92e4-4af6-b8d0-6e17a143a3b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:46.985780 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd190904-92e4-4af6-b8d0-6e17a143a3b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:46.987661 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd190904-92e4-4af6-b8d0-6e17a143a3b9 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:46.995521 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-6893b37c-8925-4e72-a030-719734776a1f None None] MFA Rules not processed for user `dbec912e4d5c4c6aabed5a98b6aa632d`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:46.997268 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6893b37c-8925-4e72-a030-719734776a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:47.012184 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1022/2045] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:47.012184 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd190904-92e4-4af6-b8d0-6e17a143a3b9 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:47.031232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd190904-92e4-4af6-b8d0-6e17a143a3b9 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:47.040378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27622744-6446-4c27-9d22-8e83d8e25ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:47.041022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27622744-6446-4c27-9d22-8e83d8e25ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:47.041303 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27622744-6446-4c27-9d22-8e83d8e25ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:47.041737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27622744-6446-4c27-9d22-8e83d8e25ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:47.042432 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27622744-6446-4c27-9d22-8e83d8e25ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:47.044157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27622744-6446-4c27-9d22-8e83d8e25ad8 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:47.061619 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1023/2046] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:47.061619 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27622744-6446-4c27-9d22-8e83d8e25ad8 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:47.071518 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27622744-6446-4c27-9d22-8e83d8e25ad8 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:47.076468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5c9ff-37ad-4007-b81b-9d2a83b0294b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:47.077158 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5c9ff-37ad-4007-b81b-9d2a83b0294b admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:47.077435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5c9ff-37ad-4007-b81b-9d2a83b0294b admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:47.077708 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5c9ff-37ad-4007-b81b-9d2a83b0294b admin admin] PATH_INFO: `/v3/projects/1937b4d39ca046b69f6dbfe888993d00/users/1a764dacb2f04783b246d7c317077757/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:47.098430 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1024/2047] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:47.099673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893b37c-8925-4e72-a030-719734776a1f None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:47.115936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6893b37c-8925-4e72-a030-719734776a1f None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:47.121954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5c9ff-37ad-4007-b81b-9d2a83b0294b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1937b4d39ca046b69f6dbfe888993d00, user_id=1a764dacb2f04783b246d7c317077757, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:47.127437 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f65bd-76d1-4f09-a50c-c648cfb9a7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:47.128710 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f65bd-76d1-4f09-a50c-c648cfb9a7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:47.128987 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f65bd-76d1-4f09-a50c-c648cfb9a7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:47.129258 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f65bd-76d1-4f09-a50c-c648cfb9a7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:47.129920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f65bd-76d1-4f09-a50c-c648cfb9a7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:47.131000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5c9ff-37ad-4007-b81b-9d2a83b0294b admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:47.131557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f65bd-76d1-4f09-a50c-c648cfb9a7c6 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:47.150308 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1025/2048] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:47.155607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f65bd-76d1-4f09-a50c-c648cfb9a7c6 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:47.157808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-531f65bd-76d1-4f09-a50c-c648cfb9a7c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:47.163742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1024/2049] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:47 2022] PUT /identity/v3/projects/1937b4d39ca046b69f6dbfe888993d00/users/1a764dacb2f04783b246d7c317077757/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:47.171395 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5c9ff-37ad-4007-b81b-9d2a83b0294b admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:47.175452 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f65bd-76d1-4f09-a50c-c648cfb9a7c6 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:47.278939 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74f56f-f354-4232-9f16-c7237e6c3836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:47.279617 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74f56f-f354-4232-9f16-c7237e6c3836 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:47.279897 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74f56f-f354-4232-9f16-c7237e6c3836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:47.280253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74f56f-f354-4232-9f16-c7237e6c3836 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:47.280883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74f56f-f354-4232-9f16-c7237e6c3836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:47.282834 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74f56f-f354-4232-9f16-c7237e6c3836 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:47.303301 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5c9ff-37ad-4007-b81b-9d2a83b0294b admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:47.311270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c4eef-2840-4a87-bde6-48a3279e8679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:47.312264 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c4eef-2840-4a87-bde6-48a3279e8679 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:47.312644 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c4eef-2840-4a87-bde6-48a3279e8679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:47.313004 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c4eef-2840-4a87-bde6-48a3279e8679 admin admin] PATH_INFO: `/v3/projects/d12435b63a424deeb1cca3ba959245a4/users/bb092d77de334d3ab631a42979460c59/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:47.325177 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1026/2050] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:27:47 2022] POST /identity/v3/projects => generated 371 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:47.325937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74f56f-f354-4232-9f16-c7237e6c3836 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:47.329910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c4eef-2840-4a87-bde6-48a3279e8679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d12435b63a424deeb1cca3ba959245a4, user_id=bb092d77de334d3ab631a42979460c59, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:47.333181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c4eef-2840-4a87-bde6-48a3279e8679 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:47.344277 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:47.349405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1025/2051] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:47 2022] PUT /identity/v3/projects/d12435b63a424deeb1cca3ba959245a4/users/bb092d77de334d3ab631a42979460c59/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:47.350273 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c4eef-2840-4a87-bde6-48a3279e8679 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:47.410641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:47.411828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74f56f-f354-4232-9f16-c7237e6c3836 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:47.428514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c4eef-2840-4a87-bde6-48a3279e8679 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:47.437480 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b00d53-b6c2-454c-aa0e-36b4d8680664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:47.438357 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b00d53-b6c2-454c-aa0e-36b4d8680664 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:47.438748 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b00d53-b6c2-454c-aa0e-36b4d8680664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:47.439199 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b00d53-b6c2-454c-aa0e-36b4d8680664 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:47.440124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b00d53-b6c2-454c-aa0e-36b4d8680664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:47.442421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b00d53-b6c2-454c-aa0e-36b4d8680664 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:47.471049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1026/2052] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:27:47 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:47.472892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf34590-7793-42b6-ba1f-49243042e7ce None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:47.472892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf34590-7793-42b6-ba1f-49243042e7ce None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:47.473069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf34590-7793-42b6-ba1f-49243042e7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:47.479647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7ba03-702a-417e-8bad-40085816cea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:47.480454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7ba03-702a-417e-8bad-40085816cea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:47.480621 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7ba03-702a-417e-8bad-40085816cea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:47.480908 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7ba03-702a-417e-8bad-40085816cea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:47.481647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7ba03-702a-417e-8bad-40085816cea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:47.483453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7ba03-702a-417e-8bad-40085816cea5 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:47.493314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-4bf34590-7793-42b6-ba1f-49243042e7ce None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:47.494426 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1027/2053] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:47.495034 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7ba03-702a-417e-8bad-40085816cea5 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:47.506089 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7ba03-702a-417e-8bad-40085816cea5 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:47.509170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4bf34590-7793-42b6-ba1f-49243042e7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:47.511028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9709b21-5ca9-46a6-95b3-b2db0a1eff95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:47.511780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9709b21-5ca9-46a6-95b3-b2db0a1eff95 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:47.512091 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9709b21-5ca9-46a6-95b3-b2db0a1eff95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:47.512425 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9709b21-5ca9-46a6-95b3-b2db0a1eff95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:47.513177 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9709b21-5ca9-46a6-95b3-b2db0a1eff95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:47.518558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9709b21-5ca9-46a6-95b3-b2db0a1eff95 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:47.523575 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1027/2054] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:47.526221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf34590-7793-42b6-ba1f-49243042e7ce None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:47.544454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1028/2055] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:47.549148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9709b21-5ca9-46a6-95b3-b2db0a1eff95 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:47.550933 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e9709b21-5ca9-46a6-95b3-b2db0a1eff95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:47.553825 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf34590-7793-42b6-ba1f-49243042e7ce None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:47.558747 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f3869-6e58-4642-89bd-af74f188af16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:47.559433 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f3869-6e58-4642-89bd-af74f188af16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:47.559733 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f3869-6e58-4642-89bd-af74f188af16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:47.560012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f3869-6e58-4642-89bd-af74f188af16 admin admin] PATH_INFO: `/v3/projects/1937b4d39ca046b69f6dbfe888993d00/users/1a764dacb2f04783b246d7c317077757/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:47.562249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f3869-6e58-4642-89bd-af74f188af16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1937b4d39ca046b69f6dbfe888993d00, user_id=1a764dacb2f04783b246d7c317077757, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:47.566655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f3869-6e58-4642-89bd-af74f188af16 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:47.584696 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1028/2056] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:47 2022] PUT /identity/v3/projects/1937b4d39ca046b69f6dbfe888993d00/users/1a764dacb2f04783b246d7c317077757/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:47.586048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f3869-6e58-4642-89bd-af74f188af16 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:47.635155 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9709b21-5ca9-46a6-95b3-b2db0a1eff95 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:47.685446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f3869-6e58-4642-89bd-af74f188af16 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:47.691019 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f1c95-a4bd-4913-a23a-e13d7d847caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:47.691655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f1c95-a4bd-4913-a23a-e13d7d847caa admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:47.692034 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f1c95-a4bd-4913-a23a-e13d7d847caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:47.692316 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f1c95-a4bd-4913-a23a-e13d7d847caa admin admin] PATH_INFO: `/v3/projects/933ad15d24bc43b8b6bbe1f1c489924f/users/7a3aa23e7ce445649eaab174603fc0a6/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:47.704648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5270e74-0ece-45b3-bd75-2016e0677725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:47.709577 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5270e74-0ece-45b3-bd75-2016e0677725 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:47.710056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5270e74-0ece-45b3-bd75-2016e0677725 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:47.710349 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5270e74-0ece-45b3-bd75-2016e0677725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:47.711122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f1c95-a4bd-4913-a23a-e13d7d847caa admin admin] RBAC: Authorizing `identity:create_grant(project_id=933ad15d24bc43b8b6bbe1f1c489924f, user_id=7a3aa23e7ce445649eaab174603fc0a6, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:47.713893 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4f1c95-a4bd-4913-a23a-e13d7d847caa admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:47.722168 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5270e74-0ece-45b3-bd75-2016e0677725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:47.723645 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5270e74-0ece-45b3-bd75-2016e0677725 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:47.732968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1029/2057] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:47 2022] PUT /identity/v3/projects/933ad15d24bc43b8b6bbe1f1c489924f/users/7a3aa23e7ce445649eaab174603fc0a6/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:47.734862 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b88eac-f7f9-4606-a0da-6c2c47d51368 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:47.735180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b88eac-f7f9-4606-a0da-6c2c47d51368 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:47.735670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b88eac-f7f9-4606-a0da-6c2c47d51368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:47.737736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1029/2058] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:47.738666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5270e74-0ece-45b3-bd75-2016e0677725 service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:47.746990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:47.753378 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5270e74-0ece-45b3-bd75-2016e0677725 service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:47.769235 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-08b88eac-f7f9-4606-a0da-6c2c47d51368 None None] MFA Rules not processed for user `1a764dacb2f04783b246d7c317077757`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:47.770681 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-08b88eac-f7f9-4606-a0da-6c2c47d51368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:47.837643 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96432401-66e4-438b-b18b-35ca6770f0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:47.837860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96432401-66e4-438b-b18b-35ca6770f0d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:47.838131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96432401-66e4-438b-b18b-35ca6770f0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:47.838458 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96432401-66e4-438b-b18b-35ca6770f0d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:47.839256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96432401-66e4-438b-b18b-35ca6770f0d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:47.842514 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96432401-66e4-438b-b18b-35ca6770f0d1 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:47.861055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1030/2059] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:47.867149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96432401-66e4-438b-b18b-35ca6770f0d1 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:47.888998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96432401-66e4-438b-b18b-35ca6770f0d1 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:47.899219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1030/2060] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:47.899871 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682f24b-f6ae-46b9-b88c-36fb1bfe6786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:47.900718 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682f24b-f6ae-46b9-b88c-36fb1bfe6786 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:47.901521 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682f24b-f6ae-46b9-b88c-36fb1bfe6786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:47.901521 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682f24b-f6ae-46b9-b88c-36fb1bfe6786 admin admin] PATH_INFO: `/v3/projects/933ad15d24bc43b8b6bbe1f1c489924f/users/7a3aa23e7ce445649eaab174603fc0a6/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:47.903907 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682f24b-f6ae-46b9-b88c-36fb1bfe6786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=933ad15d24bc43b8b6bbe1f1c489924f, user_id=7a3aa23e7ce445649eaab174603fc0a6, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:47.906979 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682f24b-f6ae-46b9-b88c-36fb1bfe6786 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:47.911516 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b88eac-f7f9-4606-a0da-6c2c47d51368 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:47.918924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-08b88eac-f7f9-4606-a0da-6c2c47d51368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:47.921483 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1031/2061] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:47 2022] PUT /identity/v3/projects/933ad15d24bc43b8b6bbe1f1c489924f/users/7a3aa23e7ce445649eaab174603fc0a6/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:47.967738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:48.012229 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b88eac-f7f9-4606-a0da-6c2c47d51368 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:48.099836 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc74b3-e1af-46e5-9233-57bfca4b9c42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:48.100497 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc74b3-e1af-46e5-9233-57bfca4b9c42 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:48.100818 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc74b3-e1af-46e5-9233-57bfca4b9c42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:48.101096 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc74b3-e1af-46e5-9233-57bfca4b9c42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:48.103161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-2edc74b3-e1af-46e5-9233-57bfca4b9c42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:48.192601 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc74b3-e1af-46e5-9233-57bfca4b9c42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:48.194244 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc74b3-e1af-46e5-9233-57bfca4b9c42 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:48.206174 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1031/2062] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:50.251106 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682f24b-f6ae-46b9-b88c-36fb1bfe6786 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:50.253596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-2682f24b-f6ae-46b9-b88c-36fb1bfe6786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:50.270801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682f24b-f6ae-46b9-b88c-36fb1bfe6786 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:50.275320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7aabd-14b5-438c-9fb3-0b7fc40532fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:50.276163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7aabd-14b5-438c-9fb3-0b7fc40532fd service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:50.276540 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7aabd-14b5-438c-9fb3-0b7fc40532fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:50.276950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7aabd-14b5-438c-9fb3-0b7fc40532fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:50.364274 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7aabd-14b5-438c-9fb3-0b7fc40532fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:50.365974 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7aabd-14b5-438c-9fb3-0b7fc40532fd service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:50.387937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1032/2063] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:53.071512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc74b3-e1af-46e5-9233-57bfca4b9c42 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:53.082151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc74b3-e1af-46e5-9233-57bfca4b9c42 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:53.086938 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6f4cf-2e4e-4d30-9b60-2517261e247e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:53.087285 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6f4cf-2e4e-4d30-9b60-2517261e247e service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:53.087647 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6f4cf-2e4e-4d30-9b60-2517261e247e service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:53.087925 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6f4cf-2e4e-4d30-9b60-2517261e247e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:53.163315 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6f4cf-2e4e-4d30-9b60-2517261e247e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:53.165139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6f4cf-2e4e-4d30-9b60-2517261e247e service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:53.181106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1032/2064] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:54.099357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7aabd-14b5-438c-9fb3-0b7fc40532fd service cinder] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:54.101710 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-eec7aabd-14b5-438c-9fb3-0b7fc40532fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:54.282679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7aabd-14b5-438c-9fb3-0b7fc40532fd service cinder] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:54.309557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306657ac-4a1b-47c1-8f49-854faa6bd93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:54.310656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306657ac-4a1b-47c1-8f49-854faa6bd93b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:54.312100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306657ac-4a1b-47c1-8f49-854faa6bd93b admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:54.314179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306657ac-4a1b-47c1-8f49-854faa6bd93b admin admin] PATH_INFO: `/v3/users/591de50d89e144da854de57be72477a2` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:54.318323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306657ac-4a1b-47c1-8f49-854faa6bd93b admin admin] RBAC: Authorizing `identity:delete_user(user_id=591de50d89e144da854de57be72477a2)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:54.321402 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306657ac-4a1b-47c1-8f49-854faa6bd93b admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:54.367479 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:54.437186 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-306657ac-4a1b-47c1-8f49-854faa6bd93b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '591de50d89e144da854de57be72477a2'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:54.472653 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-306657ac-4a1b-47c1-8f49-854faa6bd93b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '591de50d89e144da854de57be72477a2'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:54.486583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-306657ac-4a1b-47c1-8f49-854faa6bd93b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '591de50d89e144da854de57be72477a2'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:54.501288 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-306657ac-4a1b-47c1-8f49-854faa6bd93b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '591de50d89e144da854de57be72477a2'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:54.505350 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-306657ac-4a1b-47c1-8f49-854faa6bd93b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '591de50d89e144da854de57be72477a2'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:54.543262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1033/2065] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:27:54 2022] DELETE /identity/v3/users/591de50d89e144da854de57be72477a2 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:54.549742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba4aa6-b90f-4a7d-8a5a-61962685d075 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:54.551846 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba4aa6-b90f-4a7d-8a5a-61962685d075 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:54.552354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba4aa6-b90f-4a7d-8a5a-61962685d075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:54.619123 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-0cba4aa6-b90f-4a7d-8a5a-61962685d075 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:54.621115 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0cba4aa6-b90f-4a7d-8a5a-61962685d075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:54.673487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:54.773195 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1033/2066] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:54.789063 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306657ac-4a1b-47c1-8f49-854faa6bd93b admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:54.791121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-306657ac-4a1b-47c1-8f49-854faa6bd93b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:54.910230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306657ac-4a1b-47c1-8f49-854faa6bd93b admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:54.915616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3b52b-2c26-4dca-ac4a-0c21cf9464a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:54.916849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3b52b-2c26-4dca-ac4a-0c21cf9464a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:54.917349 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3b52b-2c26-4dca-ac4a-0c21cf9464a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:54.917634 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3b52b-2c26-4dca-ac4a-0c21cf9464a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:54.933454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3b52b-2c26-4dca-ac4a-0c21cf9464a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:54.935564 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3b52b-2c26-4dca-ac4a-0c21cf9464a1 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:54.952944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1034/2067] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:56.810281 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba4aa6-b90f-4a7d-8a5a-61962685d075 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:56.810281 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0cba4aa6-b90f-4a7d-8a5a-61962685d075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:56.839091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba4aa6-b90f-4a7d-8a5a-61962685d075 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:56.869202 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ff4e7-206c-4bf4-bac2-1553017f08ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:56.869910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ff4e7-206c-4bf4-bac2-1553017f08ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:56.870191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ff4e7-206c-4bf4-bac2-1553017f08ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:56.870470 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ff4e7-206c-4bf4-bac2-1553017f08ca admin admin] PATH_INFO: `/v3/projects/ec1cacb37fad4e499295edc3e317ba22` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:56.872288 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ff4e7-206c-4bf4-bac2-1553017f08ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec1cacb37fad4e499295edc3e317ba22)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:56.874276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ff4e7-206c-4bf4-bac2-1553017f08ca admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:56.933721 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382efd5b-0ce9-48ea-8df2-acbfaaba7ecd None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:56.934810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382efd5b-0ce9-48ea-8df2-acbfaaba7ecd None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:56.935842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382efd5b-0ce9-48ea-8df2-acbfaaba7ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:56.988790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-382efd5b-0ce9-48ea-8df2-acbfaaba7ecd None None] MFA Rules not processed for user `bb092d77de334d3ab631a42979460c59`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:56.990048 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-382efd5b-0ce9-48ea-8df2-acbfaaba7ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:57.026127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-a97ff4e7-206c-4bf4-bac2-1553017f08ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec1cacb37fad4e499295edc3e317ba22', 'request_id': 'req-a97ff4e7-206c-4bf4-bac2-1553017f08ca'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:57.033017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-a97ff4e7-206c-4bf4-bac2-1553017f08ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec1cacb37fad4e499295edc3e317ba22', 'request_id': 'req-a97ff4e7-206c-4bf4-bac2-1553017f08ca'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:57.040378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-a97ff4e7-206c-4bf4-bac2-1553017f08ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec1cacb37fad4e499295edc3e317ba22', 'request_id': 'req-a97ff4e7-206c-4bf4-bac2-1553017f08ca'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:57.070076 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-a97ff4e7-206c-4bf4-bac2-1553017f08ca admin admin] The token cache is being invalidate because project ec1cacb37fad4e499295edc3e317ba22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:27:57.070379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-a97ff4e7-206c-4bf4-bac2-1553017f08ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:57.071334 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-a97ff4e7-206c-4bf4-bac2-1553017f08ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:57.073493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1034/2068] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:27:56 2022] DELETE /identity/v3/projects/ec1cacb37fad4e499295edc3e317ba22 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:57.075194 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d60800-4a4c-424a-839b-a07d10416f2d None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:57.075550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d60800-4a4c-424a-839b-a07d10416f2d None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:57.075893 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d60800-4a4c-424a-839b-a07d10416f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:57.100875 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1035/2069] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:57.101678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382efd5b-0ce9-48ea-8df2-acbfaaba7ecd None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:57.104066 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-382efd5b-0ce9-48ea-8df2-acbfaaba7ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:57.126067 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382efd5b-0ce9-48ea-8df2-acbfaaba7ecd None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:57.131605 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-05d60800-4a4c-424a-839b-a07d10416f2d None None] MFA Rules not processed for user `7a3aa23e7ce445649eaab174603fc0a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:57.133536 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-05d60800-4a4c-424a-839b-a07d10416f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:57.178236 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:57.226893 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac70615-5b53-4c5d-93f4-f777d1c59a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:57.227797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac70615-5b53-4c5d-93f4-f777d1c59a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:57.228131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac70615-5b53-4c5d-93f4-f777d1c59a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:57.228417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac70615-5b53-4c5d-93f4-f777d1c59a17 admin admin] PATH_INFO: `/v3/users/1d86c0a011db4d1fa8f557b060ff90ad` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:57.229970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac70615-5b53-4c5d-93f4-f777d1c59a17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d86c0a011db4d1fa8f557b060ff90ad)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:57.233313 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac70615-5b53-4c5d-93f4-f777d1c59a17 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:57.256346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1035/2070] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:57.258174 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d60800-4a4c-424a-839b-a07d10416f2d None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:57.278199 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-05d60800-4a4c-424a-839b-a07d10416f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:57.303744 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-6ac70615-5b53-4c5d-93f4-f777d1c59a17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d86c0a011db4d1fa8f557b060ff90ad'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:57.334927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-6ac70615-5b53-4c5d-93f4-f777d1c59a17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d86c0a011db4d1fa8f557b060ff90ad'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:57.346230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-6ac70615-5b53-4c5d-93f4-f777d1c59a17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d86c0a011db4d1fa8f557b060ff90ad'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:57.346926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-6ac70615-5b53-4c5d-93f4-f777d1c59a17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d86c0a011db4d1fa8f557b060ff90ad'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:57.347783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-6ac70615-5b53-4c5d-93f4-f777d1c59a17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d86c0a011db4d1fa8f557b060ff90ad'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:57.373470 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1036/2071] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:27:57 2022] DELETE /identity/v3/users/1d86c0a011db4d1fa8f557b060ff90ad => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:57.374735 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac70615-5b53-4c5d-93f4-f777d1c59a17 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:57.376022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6ac70615-5b53-4c5d-93f4-f777d1c59a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:57.379533 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:57.537859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d60800-4a4c-424a-839b-a07d10416f2d None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:57.592668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac70615-5b53-4c5d-93f4-f777d1c59a17 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:57.605623 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efb26c-69c6-4182-8c28-667d671a6a11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:57.607114 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efb26c-69c6-4182-8c28-667d671a6a11 service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:57.607114 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efb26c-69c6-4182-8c28-667d671a6a11 service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:57.607114 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efb26c-69c6-4182-8c28-667d671a6a11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:57.608960 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-14efb26c-69c6-4182-8c28-667d671a6a11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:57.700691 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:57.706204 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efb26c-69c6-4182-8c28-667d671a6a11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:57.709572 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efb26c-69c6-4182-8c28-667d671a6a11 service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:57.721454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25e5e2-8d34-482f-aca6-0de05cfed3d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:57.723315 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25e5e2-8d34-482f-aca6-0de05cfed3d1 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:57.723794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25e5e2-8d34-482f-aca6-0de05cfed3d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:57.724357 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25e5e2-8d34-482f-aca6-0de05cfed3d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:57.725407 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1037/2072] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:57.728368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-2e25e5e2-8d34-482f-aca6-0de05cfed3d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:57.890304 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25e5e2-8d34-482f-aca6-0de05cfed3d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:57.893024 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e25e5e2-8d34-482f-aca6-0de05cfed3d1 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:57.916089 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1036/2073] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:58.034203 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efb26c-69c6-4182-8c28-667d671a6a11 service nova] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:58.035799 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-14efb26c-69c6-4182-8c28-667d671a6a11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:58.181048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25e5e2-8d34-482f-aca6-0de05cfed3d1 service nova] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:58.183089 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-2e25e5e2-8d34-482f-aca6-0de05cfed3d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:58.199716 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25e5e2-8d34-482f-aca6-0de05cfed3d1 service nova] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:58.201102 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efb26c-69c6-4182-8c28-667d671a6a11 service nova] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:58.205006 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98059de-ac8b-4cc3-af1a-eba2efca9371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:58.205713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98059de-ac8b-4cc3-af1a-eba2efca9371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:58.205998 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98059de-ac8b-4cc3-af1a-eba2efca9371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:58.206545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98059de-ac8b-4cc3-af1a-eba2efca9371 admin admin] PATH_INFO: `/v3/users/ed5522645027406abc9741967cd22501` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:58.207813 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98059de-ac8b-4cc3-af1a-eba2efca9371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed5522645027406abc9741967cd22501)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:58.210349 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98059de-ac8b-4cc3-af1a-eba2efca9371 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:58.212787 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af2bbb-6b85-47bd-8524-302ae776310a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:58.213756 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af2bbb-6b85-47bd-8524-302ae776310a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:58.214438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af2bbb-6b85-47bd-8524-302ae776310a admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:58.214910 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af2bbb-6b85-47bd-8524-302ae776310a admin admin] PATH_INFO: `/v3/projects/02b13178599e480da0ee171fee7cb3a9` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:58.216482 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af2bbb-6b85-47bd-8524-302ae776310a admin admin] RBAC: Authorizing `identity:delete_project(project_id=02b13178599e480da0ee171fee7cb3a9)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:58.218920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af2bbb-6b85-47bd-8524-302ae776310a admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:58.284450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-10af2bbb-6b85-47bd-8524-302ae776310a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02b13178599e480da0ee171fee7cb3a9', 'request_id': 'req-10af2bbb-6b85-47bd-8524-302ae776310a'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:58.287743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d98059de-ac8b-4cc3-af1a-eba2efca9371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed5522645027406abc9741967cd22501'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:58.293714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-10af2bbb-6b85-47bd-8524-302ae776310a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02b13178599e480da0ee171fee7cb3a9', 'request_id': 'req-10af2bbb-6b85-47bd-8524-302ae776310a'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:58.297517 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d98059de-ac8b-4cc3-af1a-eba2efca9371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed5522645027406abc9741967cd22501'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:58.304265 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-10af2bbb-6b85-47bd-8524-302ae776310a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02b13178599e480da0ee171fee7cb3a9', 'request_id': 'req-10af2bbb-6b85-47bd-8524-302ae776310a'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:58.310085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:58.322557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d98059de-ac8b-4cc3-af1a-eba2efca9371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed5522645027406abc9741967cd22501'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:58.323815 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d98059de-ac8b-4cc3-af1a-eba2efca9371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed5522645027406abc9741967cd22501'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:58.329892 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-10af2bbb-6b85-47bd-8524-302ae776310a admin admin] The token cache is being invalidate because project 02b13178599e480da0ee171fee7cb3a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:27:58.331451 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-10af2bbb-6b85-47bd-8524-302ae776310a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:58.332942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-10af2bbb-6b85-47bd-8524-302ae776310a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:58.335779 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1038/2074] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:27:58 2022] DELETE /identity/v3/projects/02b13178599e480da0ee171fee7cb3a9 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:58.347206 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d97b41-2bc6-4a27-a6e7-07b24ef5ffd0 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:58.347784 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d97b41-2bc6-4a27-a6e7-07b24ef5ffd0 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:58.348454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d97b41-2bc6-4a27-a6e7-07b24ef5ffd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:58.351724 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d98059de-ac8b-4cc3-af1a-eba2efca9371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed5522645027406abc9741967cd22501'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:58.378351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1037/2075] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:27:58 2022] DELETE /identity/v3/users/ed5522645027406abc9741967cd22501 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:58.384620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18ae7e-fa9d-40b8-87e5-e2f928741e92 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:58.385619 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18ae7e-fa9d-40b8-87e5-e2f928741e92 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:58.386386 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18ae7e-fa9d-40b8-87e5-e2f928741e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:58.393679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-c0d97b41-2bc6-4a27-a6e7-07b24ef5ffd0 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:58.395025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-c0d97b41-2bc6-4a27-a6e7-07b24ef5ffd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:58.424678 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-0c18ae7e-fa9d-40b8-87e5-e2f928741e92 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:58.427914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0c18ae7e-fa9d-40b8-87e5-e2f928741e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:58.501901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1039/2076] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:58.507827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fd16a-1b79-452e-b2a4-f92f8f7c38b3 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:58.509983 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fd16a-1b79-452e-b2a4-f92f8f7c38b3 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:58.510773 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731fd16a-1b79-452e-b2a4-f92f8f7c38b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:58.589887 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1038/2077] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:58.590517 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-731fd16a-1b79-452e-b2a4-f92f8f7c38b3 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:58.591798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-731fd16a-1b79-452e-b2a4-f92f8f7c38b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:58.604036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1040/2078] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:27:58.608177 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18ae7e-fa9d-40b8-87e5-e2f928741e92 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:58.619781 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731fd16a-1b79-452e-b2a4-f92f8f7c38b3 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:58.619781 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-731fd16a-1b79-452e-b2a4-f92f8f7c38b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:58.639548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18ae7e-fa9d-40b8-87e5-e2f928741e92 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:58.663623 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf3fb8-eb68-4f67-84ab-5a346df2fb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:58.664300 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf3fb8-eb68-4f67-84ab-5a346df2fb7c admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:58.664574 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf3fb8-eb68-4f67-84ab-5a346df2fb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:58.664840 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf3fb8-eb68-4f67-84ab-5a346df2fb7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:58.665294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:58.666226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf3fb8-eb68-4f67-84ab-5a346df2fb7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:27:58.666226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf3fb8-eb68-4f67-84ab-5a346df2fb7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:58.668700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf3fb8-eb68-4f67-84ab-5a346df2fb7c admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:58.694673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1039/2079] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:27:58.699128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf3fb8-eb68-4f67-84ab-5a346df2fb7c admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:58.724758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf3fb8-eb68-4f67-84ab-5a346df2fb7c admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:58.743959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc5741-ab9f-48f6-9c7a-e7abfa849eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:58.744337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc5741-ab9f-48f6-9c7a-e7abfa849eff admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:58.744616 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc5741-ab9f-48f6-9c7a-e7abfa849eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:58.744905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dc5741-ab9f-48f6-9c7a-e7abfa849eff admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:58.745567 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc5741-ab9f-48f6-9c7a-e7abfa849eff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:58.747610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dc5741-ab9f-48f6-9c7a-e7abfa849eff admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:58.752689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731fd16a-1b79-452e-b2a4-f92f8f7c38b3 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:58.756913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad0260-310b-4f95-aa3d-acac24492708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:58.757565 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad0260-310b-4f95-aa3d-acac24492708 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:58.757863 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad0260-310b-4f95-aa3d-acac24492708 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:58.758121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad0260-310b-4f95-aa3d-acac24492708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:58.775259 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad0260-310b-4f95-aa3d-acac24492708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:58.778039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad0260-310b-4f95-aa3d-acac24492708 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:58.803547 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1041/2080] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:27:58.819121 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1040/2081] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:27:58 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:58.823617 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad0260-310b-4f95-aa3d-acac24492708 service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:58.840742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad0260-310b-4f95-aa3d-acac24492708 service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:58.992902 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa7aa7-6917-4aec-baa1-e483b3cc4815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:58.993708 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa7aa7-6917-4aec-baa1-e483b3cc4815 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:58.993994 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa7aa7-6917-4aec-baa1-e483b3cc4815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:58.994278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa7aa7-6917-4aec-baa1-e483b3cc4815 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:58.996895 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa7aa7-6917-4aec-baa1-e483b3cc4815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:58.999291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa7aa7-6917-4aec-baa1-e483b3cc4815 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:59.037195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1042/2082] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:27:58 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:27:59.038451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc5741-ab9f-48f6-9c7a-e7abfa849eff admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:59.051651 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dc5741-ab9f-48f6-9c7a-e7abfa849eff admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:59.064494 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896b55b-9c54-4864-8ac6-f65dd78b1a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:59.066569 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896b55b-9c54-4864-8ac6-f65dd78b1a5f admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:59.067505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896b55b-9c54-4864-8ac6-f65dd78b1a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:59.068320 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896b55b-9c54-4864-8ac6-f65dd78b1a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:59.069817 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896b55b-9c54-4864-8ac6-f65dd78b1a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:59.075879 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896b55b-9c54-4864-8ac6-f65dd78b1a5f admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:59.103554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1041/2083] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:59.104101 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa7aa7-6917-4aec-baa1-e483b3cc4815 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:59.114186 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896b55b-9c54-4864-8ac6-f65dd78b1a5f admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:59.116934 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0896b55b-9c54-4864-8ac6-f65dd78b1a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:59.117389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa7aa7-6917-4aec-baa1-e483b3cc4815 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:59.132178 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896b55b-9c54-4864-8ac6-f65dd78b1a5f admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:59.134677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2026025-c6a0-42cf-8fa7-8b0b9542a73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:59.135593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2026025-c6a0-42cf-8fa7-8b0b9542a73b admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:59.135883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2026025-c6a0-42cf-8fa7-8b0b9542a73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:59.136574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2026025-c6a0-42cf-8fa7-8b0b9542a73b admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:59.136915 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2026025-c6a0-42cf-8fa7-8b0b9542a73b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:59.138851 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2026025-c6a0-42cf-8fa7-8b0b9542a73b admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:59.140215 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab968c-d032-47d0-9fc7-6caca3d8e4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:59.141998 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab968c-d032-47d0-9fc7-6caca3d8e4e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:59.143597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab968c-d032-47d0-9fc7-6caca3d8e4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:59.144190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab968c-d032-47d0-9fc7-6caca3d8e4e4 admin admin] PATH_INFO: `/v3/users/5fafac13941e40e8b1f95f3ac53bb241` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:59.145914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab968c-d032-47d0-9fc7-6caca3d8e4e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fafac13941e40e8b1f95f3ac53bb241)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:59.148467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab968c-d032-47d0-9fc7-6caca3d8e4e4 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:59.163840 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1043/2084] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:27:59.165797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2026025-c6a0-42cf-8fa7-8b0b9542a73b admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:59.180861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2026025-c6a0-42cf-8fa7-8b0b9542a73b admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:59.192797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf79b7-189d-4c03-9c43-fcada5e74649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:59.193488 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf79b7-189d-4c03-9c43-fcada5e74649 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:59.193769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf79b7-189d-4c03-9c43-fcada5e74649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:59.194062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf79b7-189d-4c03-9c43-fcada5e74649 admin admin] PATH_INFO: `/v3/projects/ecd1d84a80424060940081d5b2b6bf23/users/8021b97fadf4438ea002f49483461b38/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:59.242801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4dab968c-d032-47d0-9fc7-6caca3d8e4e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fafac13941e40e8b1f95f3ac53bb241'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:59.243127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf79b7-189d-4c03-9c43-fcada5e74649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecd1d84a80424060940081d5b2b6bf23, user_id=8021b97fadf4438ea002f49483461b38, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:59.246602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf79b7-189d-4c03-9c43-fcada5e74649 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:59.254748 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4dab968c-d032-47d0-9fc7-6caca3d8e4e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fafac13941e40e8b1f95f3ac53bb241'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:59.274523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4dab968c-d032-47d0-9fc7-6caca3d8e4e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fafac13941e40e8b1f95f3ac53bb241'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:59.275320 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4dab968c-d032-47d0-9fc7-6caca3d8e4e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fafac13941e40e8b1f95f3ac53bb241'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:59.287265 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4dab968c-d032-47d0-9fc7-6caca3d8e4e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fafac13941e40e8b1f95f3ac53bb241'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:27:59.290036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1044/2085] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:27:59 2022] PUT /identity/v3/projects/ecd1d84a80424060940081d5b2b6bf23/users/8021b97fadf4438ea002f49483461b38/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:59.299596 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1042/2086] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:27:59 2022] DELETE /identity/v3/users/5fafac13941e40e8b1f95f3ac53bb241 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:27:59.301606 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7002f5c-5159-438b-80c4-e5d44af90500 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:59.303824 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7002f5c-5159-438b-80c4-e5d44af90500 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:59.304124 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7002f5c-5159-438b-80c4-e5d44af90500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:59.322566 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387974e-e4a6-4504-8b2e-bd85db08c491 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:59.326033 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387974e-e4a6-4504-8b2e-bd85db08c491 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:59.326033 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387974e-e4a6-4504-8b2e-bd85db08c491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:59.375000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-a387974e-e4a6-4504-8b2e-bd85db08c491 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:59.377841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a387974e-e4a6-4504-8b2e-bd85db08c491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:59.383823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-c7002f5c-5159-438b-80c4-e5d44af90500 None None] MFA Rules not processed for user `8021b97fadf4438ea002f49483461b38`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:27:59.383823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-c7002f5c-5159-438b-80c4-e5d44af90500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:59.392328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:59.509830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1043/2087] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:59.576399 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387974e-e4a6-4504-8b2e-bd85db08c491 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:59.579437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a387974e-e4a6-4504-8b2e-bd85db08c491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:59.597855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1045/2088] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:27:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:27:59.600753 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7002f5c-5159-438b-80c4-e5d44af90500 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:59.605015 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-c7002f5c-5159-438b-80c4-e5d44af90500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:27:59.627542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:27:59.641468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7002f5c-5159-438b-80c4-e5d44af90500 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:59.649076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac723ba-740c-4e56-ad1b-efd721e78552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:59.649726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac723ba-740c-4e56-ad1b-efd721e78552 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:59.650006 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac723ba-740c-4e56-ad1b-efd721e78552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:59.650281 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac723ba-740c-4e56-ad1b-efd721e78552 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:59.650932 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac723ba-740c-4e56-ad1b-efd721e78552 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:59.656813 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac723ba-740c-4e56-ad1b-efd721e78552 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:59.745006 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1046/2089] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:27:59 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:27:59.745966 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac723ba-740c-4e56-ad1b-efd721e78552 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:59.760771 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387974e-e4a6-4504-8b2e-bd85db08c491 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:59.918703 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373985be-6b36-495c-9e95-8883e229d902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:59.920335 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373985be-6b36-495c-9e95-8883e229d902 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:59.921218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373985be-6b36-495c-9e95-8883e229d902 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:59.923471 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac723ba-740c-4e56-ad1b-efd721e78552 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:27:59.929820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373985be-6b36-495c-9e95-8883e229d902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:59.940727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283865f-53fd-495a-bb5e-e4513d68e28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:27:59.941379 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283865f-53fd-495a-bb5e-e4513d68e28d admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:27:59.941667 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283865f-53fd-495a-bb5e-e4513d68e28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:27:59.941947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1283865f-53fd-495a-bb5e-e4513d68e28d admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:27:59.945316 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283865f-53fd-495a-bb5e-e4513d68e28d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:59.946487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373985be-6b36-495c-9e95-8883e229d902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:27:59.948744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373985be-6b36-495c-9e95-8883e229d902 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:59.948920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1283865f-53fd-495a-bb5e-e4513d68e28d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:27:59.989668 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1044/2090] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:27:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:27:59.991072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373985be-6b36-495c-9e95-8883e229d902 service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:27:59.994541 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-373985be-6b36-495c-9e95-8883e229d902 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:00.009837 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1047/2091] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:27:59 2022] POST /identity/v3/users => generated 294 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:28:00.010858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283865f-53fd-495a-bb5e-e4513d68e28d admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:00.015091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373985be-6b36-495c-9e95-8883e229d902 service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:00.022183 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d7be5-5ee2-405e-8625-e9b831939816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:00.023687 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d7be5-5ee2-405e-8625-e9b831939816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:00.024162 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d7be5-5ee2-405e-8625-e9b831939816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:00.024599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d7be5-5ee2-405e-8625-e9b831939816 admin admin] PATH_INFO: `/v3/projects/349f204957e84d5f89222b84717c077a` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:00.026837 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d7be5-5ee2-405e-8625-e9b831939816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=349f204957e84d5f89222b84717c077a)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:00.035225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d7be5-5ee2-405e-8625-e9b831939816 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:00.037701 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1283865f-53fd-495a-bb5e-e4513d68e28d admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:00.048313 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039cd59-46de-4c84-a6ac-f104cc9bd1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:00.048953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039cd59-46de-4c84-a6ac-f104cc9bd1cc admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:00.049250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039cd59-46de-4c84-a6ac-f104cc9bd1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:00.049514 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039cd59-46de-4c84-a6ac-f104cc9bd1cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:00.050251 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d039cd59-46de-4c84-a6ac-f104cc9bd1cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:00.052124 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d039cd59-46de-4c84-a6ac-f104cc9bd1cc admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:00.078380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1048/2092] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:00.082726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039cd59-46de-4c84-a6ac-f104cc9bd1cc admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:00.109875 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039cd59-46de-4c84-a6ac-f104cc9bd1cc admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:00.113472 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6c5d7be5-5ee2-405e-8625-e9b831939816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '349f204957e84d5f89222b84717c077a', 'request_id': 'req-6c5d7be5-5ee2-405e-8625-e9b831939816'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:00.124593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6c5d7be5-5ee2-405e-8625-e9b831939816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '349f204957e84d5f89222b84717c077a', 'request_id': 'req-6c5d7be5-5ee2-405e-8625-e9b831939816'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:00.137616 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6c5d7be5-5ee2-405e-8625-e9b831939816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '349f204957e84d5f89222b84717c077a', 'request_id': 'req-6c5d7be5-5ee2-405e-8625-e9b831939816'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:00.141669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:00.164288 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6c5d7be5-5ee2-405e-8625-e9b831939816 admin admin] The token cache is being invalidate because project 349f204957e84d5f89222b84717c077a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:28:00.164579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6c5d7be5-5ee2-405e-8625-e9b831939816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:00.168280 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6c5d7be5-5ee2-405e-8625-e9b831939816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:00.176090 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1045/2093] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:27:59 2022] DELETE /identity/v3/projects/349f204957e84d5f89222b84717c077a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:00.181107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d7be5-5ee2-405e-8625-e9b831939816 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:00.185353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-6c5d7be5-5ee2-405e-8625-e9b831939816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:00.299931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee9b57-9548-446a-95ea-0221cfbae8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:00.300661 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee9b57-9548-446a-95ea-0221cfbae8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:00.300941 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee9b57-9548-446a-95ea-0221cfbae8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:00.303433 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee9b57-9548-446a-95ea-0221cfbae8a5 admin admin] PATH_INFO: `/v3/projects/24c0f4f2507848b9947c5608ceb15bec/users/d78dc233e52c442a84f9bd684a79d47c/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:00.337019 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee9b57-9548-446a-95ea-0221cfbae8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c0f4f2507848b9947c5608ceb15bec, user_id=d78dc233e52c442a84f9bd684a79d47c, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:00.342485 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee9b57-9548-446a-95ea-0221cfbae8a5 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:00.361489 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d7be5-5ee2-405e-8625-e9b831939816 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:00.367889 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987de988-151f-404c-8538-55aedd2afa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:00.368645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987de988-151f-404c-8538-55aedd2afa8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:00.368985 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987de988-151f-404c-8538-55aedd2afa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:00.369282 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987de988-151f-404c-8538-55aedd2afa8d admin admin] PATH_INFO: `/v3/users/43abbe23782e462080db18f07d2298d7` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:00.370619 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987de988-151f-404c-8538-55aedd2afa8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=43abbe23782e462080db18f07d2298d7)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:00.373021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987de988-151f-404c-8538-55aedd2afa8d admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:00.375022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1049/2094] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:28:00 2022] PUT /identity/v3/projects/24c0f4f2507848b9947c5608ceb15bec/users/d78dc233e52c442a84f9bd684a79d47c/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:28:00.376118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee9b57-9548-446a-95ea-0221cfbae8a5 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:00.380658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6aee9b57-9548-446a-95ea-0221cfbae8a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:00.462717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:00.475670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-987de988-151f-404c-8538-55aedd2afa8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43abbe23782e462080db18f07d2298d7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:00.498565 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-987de988-151f-404c-8538-55aedd2afa8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43abbe23782e462080db18f07d2298d7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:00.511799 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee9b57-9548-446a-95ea-0221cfbae8a5 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:00.526635 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf442ac2-beb1-4202-b275-06d4a458e8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:00.529447 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf442ac2-beb1-4202-b275-06d4a458e8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:00.529728 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf442ac2-beb1-4202-b275-06d4a458e8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:00.530021 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf442ac2-beb1-4202-b275-06d4a458e8b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:00.530711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf442ac2-beb1-4202-b275-06d4a458e8b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:00.533573 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-987de988-151f-404c-8538-55aedd2afa8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43abbe23782e462080db18f07d2298d7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:00.535160 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-987de988-151f-404c-8538-55aedd2afa8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43abbe23782e462080db18f07d2298d7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:00.536168 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf442ac2-beb1-4202-b275-06d4a458e8b9 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:00.561557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1050/2095] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:00.562154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-987de988-151f-404c-8538-55aedd2afa8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43abbe23782e462080db18f07d2298d7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:00.574886 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1046/2096] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:28:00 2022] DELETE /identity/v3/users/43abbe23782e462080db18f07d2298d7 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:00.576036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf442ac2-beb1-4202-b275-06d4a458e8b9 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:00.577910 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-bf442ac2-beb1-4202-b275-06d4a458e8b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:00.593178 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca179f-189a-4db0-b0f7-07276ab51802 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:00.593469 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca179f-189a-4db0-b0f7-07276ab51802 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:00.593749 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca179f-189a-4db0-b0f7-07276ab51802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:00.633196 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf442ac2-beb1-4202-b275-06d4a458e8b9 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:00.661396 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-63ca179f-189a-4db0-b0f7-07276ab51802 None None] MFA Rules not processed for user `dbec912e4d5c4c6aabed5a98b6aa632d`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:00.670032 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-63ca179f-189a-4db0-b0f7-07276ab51802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:00.876923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8dc741-0d3c-4351-9fff-574502b84737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:00.878099 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dc741-0d3c-4351-9fff-574502b84737 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:00.878099 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dc741-0d3c-4351-9fff-574502b84737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:00.878357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8dc741-0d3c-4351-9fff-574502b84737 admin admin] PATH_INFO: `/v3/projects/24c0f4f2507848b9947c5608ceb15bec/users/d78dc233e52c442a84f9bd684a79d47c/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:00.883840 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8dc741-0d3c-4351-9fff-574502b84737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c0f4f2507848b9947c5608ceb15bec, user_id=d78dc233e52c442a84f9bd684a79d47c, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:00.896844 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8dc741-0d3c-4351-9fff-574502b84737 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:00.911948 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1047/2097] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:28:00.913602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d24bed-d17b-4e0d-a909-d062b78cd4d3 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:00.913892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d24bed-d17b-4e0d-a909-d062b78cd4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:00.914165 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d24bed-d17b-4e0d-a909-d062b78cd4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:00.922491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1051/2098] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:28:00 2022] PUT /identity/v3/projects/24c0f4f2507848b9947c5608ceb15bec/users/d78dc233e52c442a84f9bd684a79d47c/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:00.936735 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ed6ba-a28a-4d3d-a9af-7c8f564621ce None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:00.937168 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ed6ba-a28a-4d3d-a9af-7c8f564621ce None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:00.937545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ed6ba-a28a-4d3d-a9af-7c8f564621ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:00.954078 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-19d24bed-d17b-4e0d-a909-d062b78cd4d3 None None] MFA Rules not processed for user `bb092d77de334d3ab631a42979460c59`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:00.955510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-19d24bed-d17b-4e0d-a909-d062b78cd4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:00.974352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:01.048611 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-b60ed6ba-a28a-4d3d-a9af-7c8f564621ce None None] MFA Rules not processed for user `d78dc233e52c442a84f9bd684a79d47c`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:01.050410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b60ed6ba-a28a-4d3d-a9af-7c8f564621ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:01.121564 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1048/2099] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:01.122508 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d24bed-d17b-4e0d-a909-d062b78cd4d3 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:01.128022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-19d24bed-d17b-4e0d-a909-d062b78cd4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:01.216996 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1052/2100] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:28:01.226351 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ed6ba-a28a-4d3d-a9af-7c8f564621ce None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:01.231557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b60ed6ba-a28a-4d3d-a9af-7c8f564621ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:01.310437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d24bed-d17b-4e0d-a909-d062b78cd4d3 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:01.322770 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ed6ba-a28a-4d3d-a9af-7c8f564621ce None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:01.326011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39d0eb-848f-432f-9fc3-3dbb261796fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:01.326695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39d0eb-848f-432f-9fc3-3dbb261796fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:01.326982 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39d0eb-848f-432f-9fc3-3dbb261796fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:01.327260 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39d0eb-848f-432f-9fc3-3dbb261796fd admin admin] PATH_INFO: `/v3/projects/51c4dbc7057b463b9569e16bd3f86ff4` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:01.327949 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581d3f0-6c6b-40d2-8026-90b8b4f51560 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:01.328872 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581d3f0-6c6b-40d2-8026-90b8b4f51560 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:01.329192 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39d0eb-848f-432f-9fc3-3dbb261796fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=51c4dbc7057b463b9569e16bd3f86ff4)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:01.329693 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581d3f0-6c6b-40d2-8026-90b8b4f51560 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:01.330306 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581d3f0-6c6b-40d2-8026-90b8b4f51560 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:01.331414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39d0eb-848f-432f-9fc3-3dbb261796fd admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:01.332672 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6581d3f0-6c6b-40d2-8026-90b8b4f51560 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:01.363814 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:01.452370 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2f39d0eb-848f-432f-9fc3-3dbb261796fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51c4dbc7057b463b9569e16bd3f86ff4', 'request_id': 'req-2f39d0eb-848f-432f-9fc3-3dbb261796fd'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:01.473665 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2f39d0eb-848f-432f-9fc3-3dbb261796fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51c4dbc7057b463b9569e16bd3f86ff4', 'request_id': 'req-2f39d0eb-848f-432f-9fc3-3dbb261796fd'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:01.482839 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6581d3f0-6c6b-40d2-8026-90b8b4f51560 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:01.484538 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6581d3f0-6c6b-40d2-8026-90b8b4f51560 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:01.503361 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2f39d0eb-848f-432f-9fc3-3dbb261796fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51c4dbc7057b463b9569e16bd3f86ff4', 'request_id': 'req-2f39d0eb-848f-432f-9fc3-3dbb261796fd'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:01.520319 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1053/2101] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:28:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:28:01.535404 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2f39d0eb-848f-432f-9fc3-3dbb261796fd admin admin] The token cache is being invalidate because project 51c4dbc7057b463b9569e16bd3f86ff4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:28:01.535769 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2f39d0eb-848f-432f-9fc3-3dbb261796fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:01.539024 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2f39d0eb-848f-432f-9fc3-3dbb261796fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:01.542763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1049/2102] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:28:01 2022] DELETE /identity/v3/projects/51c4dbc7057b463b9569e16bd3f86ff4 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:01.548082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81df0a0-b53e-4039-b588-d0dd57734e87 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:01.548651 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81df0a0-b53e-4039-b588-d0dd57734e87 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:01.550113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81df0a0-b53e-4039-b588-d0dd57734e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:01.600557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-f81df0a0-b53e-4039-b588-d0dd57734e87 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:01.602230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f81df0a0-b53e-4039-b588-d0dd57734e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:01.638767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39d0eb-848f-432f-9fc3-3dbb261796fd admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:01.640622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-2f39d0eb-848f-432f-9fc3-3dbb261796fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:01.781393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1054/2103] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:01.792245 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319bd9f-4389-4765-9ebc-dc450d09879c None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:01.792245 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319bd9f-4389-4765-9ebc-dc450d09879c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:01.792245 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a319bd9f-4389-4765-9ebc-dc450d09879c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:01.828195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-a319bd9f-4389-4765-9ebc-dc450d09879c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:01.830212 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-a319bd9f-4389-4765-9ebc-dc450d09879c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:01.834603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:01.848360 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1055/2104] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:01.853905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319bd9f-4389-4765-9ebc-dc450d09879c None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:01.864807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39d0eb-848f-432f-9fc3-3dbb261796fd admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:01.866310 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a319bd9f-4389-4765-9ebc-dc450d09879c None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:01.873925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee889e4e-a653-46bd-b4e7-cbc1bf0c4ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:01.874195 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dd758-898d-4af2-b288-c418fbad5c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:01.874873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dd758-898d-4af2-b288-c418fbad5c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:01.875583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee889e4e-a653-46bd-b4e7-cbc1bf0c4ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:01.875792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dd758-898d-4af2-b288-c418fbad5c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:01.876172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0dd758-898d-4af2-b288-c418fbad5c08 admin admin] PATH_INFO: `/v3/projects/a247126043d847cd99ffd80841064252` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:01.876354 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee889e4e-a653-46bd-b4e7-cbc1bf0c4ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:01.876875 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee889e4e-a653-46bd-b4e7-cbc1bf0c4ba9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:01.877391 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dd758-898d-4af2-b288-c418fbad5c08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a247126043d847cd99ffd80841064252)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:01.877878 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee889e4e-a653-46bd-b4e7-cbc1bf0c4ba9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:28:01.878520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee889e4e-a653-46bd-b4e7-cbc1bf0c4ba9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:01.881249 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee889e4e-a653-46bd-b4e7-cbc1bf0c4ba9 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:01.883795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0dd758-898d-4af2-b288-c418fbad5c08 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:01.898080 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1056/2105] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:28:01.903394 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee889e4e-a653-46bd-b4e7-cbc1bf0c4ba9 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:01.918022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee889e4e-a653-46bd-b4e7-cbc1bf0c4ba9 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:01.924752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873fc90-eeba-47e9-b6ed-305f53698dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:01.925820 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873fc90-eeba-47e9-b6ed-305f53698dfe admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:01.926113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873fc90-eeba-47e9-b6ed-305f53698dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:01.926393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6873fc90-eeba-47e9-b6ed-305f53698dfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:01.927056 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6873fc90-eeba-47e9-b6ed-305f53698dfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:01.929961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6873fc90-eeba-47e9-b6ed-305f53698dfe admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:01.968284 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-eb0dd758-898d-4af2-b288-c418fbad5c08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a247126043d847cd99ffd80841064252', 'request_id': 'req-eb0dd758-898d-4af2-b288-c418fbad5c08'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:01.972235 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1057/2106] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:28:01 2022] POST /identity/v3/projects => generated 365 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:28:01.973646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873fc90-eeba-47e9-b6ed-305f53698dfe admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:01.980657 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-eb0dd758-898d-4af2-b288-c418fbad5c08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a247126043d847cd99ffd80841064252', 'request_id': 'req-eb0dd758-898d-4af2-b288-c418fbad5c08'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:01.989909 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-eb0dd758-898d-4af2-b288-c418fbad5c08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a247126043d847cd99ffd80841064252', 'request_id': 'req-eb0dd758-898d-4af2-b288-c418fbad5c08'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:02.011475 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-eb0dd758-898d-4af2-b288-c418fbad5c08 admin admin] The token cache is being invalidate because project a247126043d847cd99ffd80841064252 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:28:02.013220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-eb0dd758-898d-4af2-b288-c418fbad5c08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:02.016306 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-eb0dd758-898d-4af2-b288-c418fbad5c08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:02.019685 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1050/2107] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:28:01 2022] DELETE /identity/v3/projects/a247126043d847cd99ffd80841064252 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:02.025276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dd758-898d-4af2-b288-c418fbad5c08 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:02.034466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-eb0dd758-898d-4af2-b288-c418fbad5c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:02.190072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6873fc90-eeba-47e9-b6ed-305f53698dfe admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:02.217306 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b3011-16f0-49d2-b940-95213e89e417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:02.218334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b3011-16f0-49d2-b940-95213e89e417 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:02.221190 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b3011-16f0-49d2-b940-95213e89e417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:02.225617 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b3011-16f0-49d2-b940-95213e89e417 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:02.228070 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b3011-16f0-49d2-b940-95213e89e417 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:02.235738 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1b3011-16f0-49d2-b940-95213e89e417 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:02.295738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0dd758-898d-4af2-b288-c418fbad5c08 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:02.301493 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1058/2108] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:28:01 2022] POST /identity/v3/users => generated 283 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:28:02.305919 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b3011-16f0-49d2-b940-95213e89e417 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:02.318865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-bc1b3011-16f0-49d2-b940-95213e89e417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:02.319252 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd89f6-7bd6-476c-8575-852c52ef95da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:02.319252 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd89f6-7bd6-476c-8575-852c52ef95da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:02.319252 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd89f6-7bd6-476c-8575-852c52ef95da admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:02.319252 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd89f6-7bd6-476c-8575-852c52ef95da admin admin] PATH_INFO: `/v3/users/1a764dacb2f04783b246d7c317077757` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:02.319601 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd89f6-7bd6-476c-8575-852c52ef95da admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a764dacb2f04783b246d7c317077757)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:02.319601 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd89f6-7bd6-476c-8575-852c52ef95da admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:02.333690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:02.345550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b3011-16f0-49d2-b940-95213e89e417 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:02.362004 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446ff69-0d84-419a-91c1-3e6fcbbe2134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:02.363231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446ff69-0d84-419a-91c1-3e6fcbbe2134 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:02.363514 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446ff69-0d84-419a-91c1-3e6fcbbe2134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:02.363789 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c446ff69-0d84-419a-91c1-3e6fcbbe2134 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:02.364892 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446ff69-0d84-419a-91c1-3e6fcbbe2134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:02.367920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c446ff69-0d84-419a-91c1-3e6fcbbe2134 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:02.397274 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1059/2109] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:02.403086 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446ff69-0d84-419a-91c1-3e6fcbbe2134 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:02.431173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-68dd89f6-7bd6-476c-8575-852c52ef95da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a764dacb2f04783b246d7c317077757'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:02.432556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c446ff69-0d84-419a-91c1-3e6fcbbe2134 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:02.447905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe244d-473f-4525-ae2f-384f8cb94382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:02.449499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe244d-473f-4525-ae2f-384f8cb94382 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:02.450072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe244d-473f-4525-ae2f-384f8cb94382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:02.450349 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe244d-473f-4525-ae2f-384f8cb94382 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:02.452251 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe244d-473f-4525-ae2f-384f8cb94382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:02.454771 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-68dd89f6-7bd6-476c-8575-852c52ef95da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a764dacb2f04783b246d7c317077757'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:02.458198 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe244d-473f-4525-ae2f-384f8cb94382 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:02.491757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1060/2110] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:02.493563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-68dd89f6-7bd6-476c-8575-852c52ef95da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a764dacb2f04783b246d7c317077757'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:02.495653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-68dd89f6-7bd6-476c-8575-852c52ef95da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a764dacb2f04783b246d7c317077757'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:02.497114 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe244d-473f-4525-ae2f-384f8cb94382 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:02.525757 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-68dd89f6-7bd6-476c-8575-852c52ef95da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a764dacb2f04783b246d7c317077757'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:02.538958 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1051/2111] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:28:02 2022] DELETE /identity/v3/users/1a764dacb2f04783b246d7c317077757 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:02.539120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe244d-473f-4525-ae2f-384f8cb94382 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:02.639701 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e3bea-05a0-48f6-81e4-0a53dcfddaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:02.640447 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e3bea-05a0-48f6-81e4-0a53dcfddaa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:02.640765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e3bea-05a0-48f6-81e4-0a53dcfddaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:02.641349 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e3bea-05a0-48f6-81e4-0a53dcfddaa7 admin admin] PATH_INFO: `/v3/projects/a9f8bd29a3874d7ca3f48ecaead0a62e/users/3ac5c1df8c174792a45e3637b6262175/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:02.665557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e3bea-05a0-48f6-81e4-0a53dcfddaa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9f8bd29a3874d7ca3f48ecaead0a62e, user_id=3ac5c1df8c174792a45e3637b6262175, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:02.669041 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e3bea-05a0-48f6-81e4-0a53dcfddaa7 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:02.696950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1061/2112] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:28:02 2022] PUT /identity/v3/projects/a9f8bd29a3874d7ca3f48ecaead0a62e/users/3ac5c1df8c174792a45e3637b6262175/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:02.707930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a68d87-8509-4f6d-89fe-78dd6c0fa54e None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:02.708326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a68d87-8509-4f6d-89fe-78dd6c0fa54e None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:02.708510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a68d87-8509-4f6d-89fe-78dd6c0fa54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:02.722772 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:02.825074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-89a68d87-8509-4f6d-89fe-78dd6c0fa54e None None] MFA Rules not processed for user `3ac5c1df8c174792a45e3637b6262175`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:02.825286 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-89a68d87-8509-4f6d-89fe-78dd6c0fa54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:03.058181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1052/2113] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:03.081184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e3bea-05a0-48f6-81e4-0a53dcfddaa7 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:03.084584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b06e3bea-05a0-48f6-81e4-0a53dcfddaa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:03.248348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e3bea-05a0-48f6-81e4-0a53dcfddaa7 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:03.261109 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a68d87-8509-4f6d-89fe-78dd6c0fa54e None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:03.262896 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-89a68d87-8509-4f6d-89fe-78dd6c0fa54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:03.268487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581c175-a5db-4737-8a5e-6897b2ead22c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:03.269097 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581c175-a5db-4737-8a5e-6897b2ead22c service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:03.269419 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581c175-a5db-4737-8a5e-6897b2ead22c service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:03.269759 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581c175-a5db-4737-8a5e-6897b2ead22c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:03.299839 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581c175-a5db-4737-8a5e-6897b2ead22c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:03.301490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581c175-a5db-4737-8a5e-6897b2ead22c service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:03.322337 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1062/2114] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:28:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:28:03.429740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a68d87-8509-4f6d-89fe-78dd6c0fa54e None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:03.442429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72763af8-0a7f-4d8f-aebe-807f4517d9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:03.445182 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72763af8-0a7f-4d8f-aebe-807f4517d9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:03.447168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72763af8-0a7f-4d8f-aebe-807f4517d9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:03.447771 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72763af8-0a7f-4d8f-aebe-807f4517d9aa admin admin] PATH_INFO: `/v3/projects/1937b4d39ca046b69f6dbfe888993d00` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:03.451734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72763af8-0a7f-4d8f-aebe-807f4517d9aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1937b4d39ca046b69f6dbfe888993d00)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:03.455080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72763af8-0a7f-4d8f-aebe-807f4517d9aa admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:03.564521 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-72763af8-0a7f-4d8f-aebe-807f4517d9aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1937b4d39ca046b69f6dbfe888993d00', 'request_id': 'req-72763af8-0a7f-4d8f-aebe-807f4517d9aa'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:03.579740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-72763af8-0a7f-4d8f-aebe-807f4517d9aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1937b4d39ca046b69f6dbfe888993d00', 'request_id': 'req-72763af8-0a7f-4d8f-aebe-807f4517d9aa'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:03.587871 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-72763af8-0a7f-4d8f-aebe-807f4517d9aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1937b4d39ca046b69f6dbfe888993d00', 'request_id': 'req-72763af8-0a7f-4d8f-aebe-807f4517d9aa'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:03.628796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-72763af8-0a7f-4d8f-aebe-807f4517d9aa admin admin] The token cache is being invalidate because project 1937b4d39ca046b69f6dbfe888993d00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:28:03.629353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-72763af8-0a7f-4d8f-aebe-807f4517d9aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:03.630425 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-72763af8-0a7f-4d8f-aebe-807f4517d9aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:03.632776 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1053/2115] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:28:03 2022] DELETE /identity/v3/projects/1937b4d39ca046b69f6dbfe888993d00 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:28:03.639352 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2762db-b671-43ac-92f9-d43cd60849a0 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:03.641334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2762db-b671-43ac-92f9-d43cd60849a0 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:03.641633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2762db-b671-43ac-92f9-d43cd60849a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:03.670009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-7b2762db-b671-43ac-92f9-d43cd60849a0 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:03.672133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-7b2762db-b671-43ac-92f9-d43cd60849a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:03.898414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1063/2116] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:03.901994 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6a523-e78e-4d8e-b5ba-33608005b8c2 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:03.901994 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6a523-e78e-4d8e-b5ba-33608005b8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:03.902379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6a523-e78e-4d8e-b5ba-33608005b8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:03.975758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-cbc6a523-e78e-4d8e-b5ba-33608005b8c2 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:03.981151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-cbc6a523-e78e-4d8e-b5ba-33608005b8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:03.999917 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1054/2117] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:04.001302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2762db-b671-43ac-92f9-d43cd60849a0 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:04.031909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2762db-b671-43ac-92f9-d43cd60849a0 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:04.050007 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4926ab-d9a1-4763-a6ed-ca07cd6e17c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:04.050754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4926ab-d9a1-4763-a6ed-ca07cd6e17c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:04.051111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4926ab-d9a1-4763-a6ed-ca07cd6e17c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:04.051457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4926ab-d9a1-4763-a6ed-ca07cd6e17c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:04.052241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4926ab-d9a1-4763-a6ed-ca07cd6e17c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:28:04.053742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4926ab-d9a1-4763-a6ed-ca07cd6e17c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:04.055484 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4926ab-d9a1-4763-a6ed-ca07cd6e17c2 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:04.058654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:04.106893 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1064/2118] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:28:04.110044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6a523-e78e-4d8e-b5ba-33608005b8c2 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:04.134307 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6a523-e78e-4d8e-b5ba-33608005b8c2 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:04.147015 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867094a1-bd34-410d-b43f-cb4d2b88c5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:04.147787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867094a1-bd34-410d-b43f-cb4d2b88c5d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:04.148289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867094a1-bd34-410d-b43f-cb4d2b88c5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:04.148662 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867094a1-bd34-410d-b43f-cb4d2b88c5d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:04.149651 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867094a1-bd34-410d-b43f-cb4d2b88c5d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:04.152590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867094a1-bd34-410d-b43f-cb4d2b88c5d7 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:04.211218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:04.242004 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1055/2119] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:28:04 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:28:04.246864 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4926ab-d9a1-4763-a6ed-ca07cd6e17c2 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:04.264564 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4926ab-d9a1-4763-a6ed-ca07cd6e17c2 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:04.406851 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c8ddd-4502-407c-a294-e760037dc726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:04.407606 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c8ddd-4502-407c-a294-e760037dc726 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:04.407966 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c8ddd-4502-407c-a294-e760037dc726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:04.408746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c8ddd-4502-407c-a294-e760037dc726 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:04.409595 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c8ddd-4502-407c-a294-e760037dc726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:04.411735 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c8ddd-4502-407c-a294-e760037dc726 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:04.444257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1065/2120] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:28:04 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:28:04.448880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867094a1-bd34-410d-b43f-cb4d2b88c5d7 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:04.468824 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867094a1-bd34-410d-b43f-cb4d2b88c5d7 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:04.474137 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec5a06-739d-4fd8-94c8-95eb197f2545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:04.474903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec5a06-739d-4fd8-94c8-95eb197f2545 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:04.475242 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec5a06-739d-4fd8-94c8-95eb197f2545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:04.475583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec5a06-739d-4fd8-94c8-95eb197f2545 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:04.476455 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec5a06-739d-4fd8-94c8-95eb197f2545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:04.478468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec5a06-739d-4fd8-94c8-95eb197f2545 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:04.498177 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c8ddd-4502-407c-a294-e760037dc726 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:04.498855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1056/2121] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:04.511192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c8ddd-4502-407c-a294-e760037dc726 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:04.519503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fee3d85-c052-48f4-8625-ec6426ba72cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:04.520621 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee3d85-c052-48f4-8625-ec6426ba72cf admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:04.521390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee3d85-c052-48f4-8625-ec6426ba72cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:04.521854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fee3d85-c052-48f4-8625-ec6426ba72cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:04.522734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee3d85-c052-48f4-8625-ec6426ba72cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:04.524861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fee3d85-c052-48f4-8625-ec6426ba72cf admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:04.545014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1066/2122] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:28:04.547255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec5a06-739d-4fd8-94c8-95eb197f2545 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:04.560716 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec5a06-739d-4fd8-94c8-95eb197f2545 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:04.565641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3749265-61d5-4f49-abc3-e85a2feae0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:04.566952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3749265-61d5-4f49-abc3-e85a2feae0c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:04.566952 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3749265-61d5-4f49-abc3-e85a2feae0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:04.567165 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3749265-61d5-4f49-abc3-e85a2feae0c0 admin admin] PATH_INFO: `/v3/projects/3757af7375eb4492b5b0288438948837/users/410f3be0a16b466b8568b980fc0c5ab7/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:04.585157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3749265-61d5-4f49-abc3-e85a2feae0c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3757af7375eb4492b5b0288438948837, user_id=410f3be0a16b466b8568b980fc0c5ab7, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:04.588382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3749265-61d5-4f49-abc3-e85a2feae0c0 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:04.611376 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1057/2123] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:28:04 2022] PUT /identity/v3/projects/3757af7375eb4492b5b0288438948837/users/410f3be0a16b466b8568b980fc0c5ab7/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:04.622204 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d432cf-235c-4328-a931-f54e51d7280f None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:04.622365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d432cf-235c-4328-a931-f54e51d7280f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:04.622646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d432cf-235c-4328-a931-f54e51d7280f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:04.656811 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-04d432cf-235c-4328-a931-f54e51d7280f None None] MFA Rules not processed for user `410f3be0a16b466b8568b980fc0c5ab7`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:04.658054 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-04d432cf-235c-4328-a931-f54e51d7280f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:04.729603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1067/2124] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:04.741526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3749265-61d5-4f49-abc3-e85a2feae0c0 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:04.743893 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f3749265-61d5-4f49-abc3-e85a2feae0c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:04.815846 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3749265-61d5-4f49-abc3-e85a2feae0c0 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:04.820102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c8d37-cb2b-463e-906e-547ea30a76a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:04.820996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c8d37-cb2b-463e-906e-547ea30a76a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:04.821295 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c8d37-cb2b-463e-906e-547ea30a76a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:04.821580 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c8d37-cb2b-463e-906e-547ea30a76a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:04.833261 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c8d37-cb2b-463e-906e-547ea30a76a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:04.835006 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c8d37-cb2b-463e-906e-547ea30a76a8 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:04.846673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1058/2125] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:28:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:28:07.951165 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d432cf-235c-4328-a931-f54e51d7280f None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:07.953826 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-04d432cf-235c-4328-a931-f54e51d7280f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:07.968949 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d432cf-235c-4328-a931-f54e51d7280f None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:07.977377 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969fa72-ce08-4c49-9e60-c6ab8514f997 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:07.978446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969fa72-ce08-4c49-9e60-c6ab8514f997 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:07.978926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969fa72-ce08-4c49-9e60-c6ab8514f997 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:07.979207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969fa72-ce08-4c49-9e60-c6ab8514f997 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:07.982787 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9969fa72-ce08-4c49-9e60-c6ab8514f997 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:08.075269 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:08.078000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969fa72-ce08-4c49-9e60-c6ab8514f997 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:08.079688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969fa72-ce08-4c49-9e60-c6ab8514f997 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:08.091744 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1068/2126] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:28:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:28:12.817963 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c8d37-cb2b-463e-906e-547ea30a76a8 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:12.893996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c8d37-cb2b-463e-906e-547ea30a76a8 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:12.899977 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3173e6-bfbc-44ee-81d3-04339707094a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:12.900653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3173e6-bfbc-44ee-81d3-04339707094a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:12.900990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3173e6-bfbc-44ee-81d3-04339707094a admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:12.901218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3173e6-bfbc-44ee-81d3-04339707094a admin admin] PATH_INFO: `/v3/users/410f3be0a16b466b8568b980fc0c5ab7` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:12.902694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3173e6-bfbc-44ee-81d3-04339707094a admin admin] RBAC: Authorizing `identity:delete_user(user_id=410f3be0a16b466b8568b980fc0c5ab7)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:12.904682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3173e6-bfbc-44ee-81d3-04339707094a admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:12.966297 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0d3173e6-bfbc-44ee-81d3-04339707094a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '410f3be0a16b466b8568b980fc0c5ab7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:12.975182 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0d3173e6-bfbc-44ee-81d3-04339707094a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '410f3be0a16b466b8568b980fc0c5ab7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:12.984516 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:INSERT=2,keystone:SELECT=90,keystone:DELETE=8 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:12.994570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0d3173e6-bfbc-44ee-81d3-04339707094a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '410f3be0a16b466b8568b980fc0c5ab7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:12.995675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0d3173e6-bfbc-44ee-81d3-04339707094a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '410f3be0a16b466b8568b980fc0c5ab7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:13.005529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0d3173e6-bfbc-44ee-81d3-04339707094a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '410f3be0a16b466b8568b980fc0c5ab7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:13.019360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1059/2127] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:28:12 2022] DELETE /identity/v3/users/410f3be0a16b466b8568b980fc0c5ab7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:13.023961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3874fae-926d-4572-b429-bfc0e6d32019 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:13.024479 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3874fae-926d-4572-b429-bfc0e6d32019 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:13.024559 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3874fae-926d-4572-b429-bfc0e6d32019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:13.043769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-a3874fae-926d-4572-b429-bfc0e6d32019 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:13.044977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-a3874fae-926d-4572-b429-bfc0e6d32019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:13.108498 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1069/2128] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:28:13.118792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3173e6-bfbc-44ee-81d3-04339707094a admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:13.120423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0d3173e6-bfbc-44ee-81d3-04339707094a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:13.185821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3173e6-bfbc-44ee-81d3-04339707094a admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:13.190328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c054f-4e8c-409a-bf3d-4937dd4a4321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:13.191059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c054f-4e8c-409a-bf3d-4937dd4a4321 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:13.191478 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c054f-4e8c-409a-bf3d-4937dd4a4321 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:13.191970 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c054f-4e8c-409a-bf3d-4937dd4a4321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:13.202999 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c054f-4e8c-409a-bf3d-4937dd4a4321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:13.204514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c054f-4e8c-409a-bf3d-4937dd4a4321 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:13.215068 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1060/2129] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:28:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:28:13.715330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3874fae-926d-4572-b429-bfc0e6d32019 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:13.718367 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-a3874fae-926d-4572-b429-bfc0e6d32019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:13.751966 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3874fae-926d-4572-b429-bfc0e6d32019 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:13.776177 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d79e90-6fd8-41fc-9bb9-38b9ca2688be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:13.776962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d79e90-6fd8-41fc-9bb9-38b9ca2688be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:13.777275 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d79e90-6fd8-41fc-9bb9-38b9ca2688be admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:13.777644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d79e90-6fd8-41fc-9bb9-38b9ca2688be admin admin] PATH_INFO: `/v3/projects/3757af7375eb4492b5b0288438948837` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:13.780244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d79e90-6fd8-41fc-9bb9-38b9ca2688be admin admin] RBAC: Authorizing `identity:delete_project(project_id=3757af7375eb4492b5b0288438948837)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:13.782525 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d79e90-6fd8-41fc-9bb9-38b9ca2688be admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:13.928142 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-80d79e90-6fd8-41fc-9bb9-38b9ca2688be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3757af7375eb4492b5b0288438948837', 'request_id': 'req-80d79e90-6fd8-41fc-9bb9-38b9ca2688be'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:13.937943 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-80d79e90-6fd8-41fc-9bb9-38b9ca2688be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3757af7375eb4492b5b0288438948837', 'request_id': 'req-80d79e90-6fd8-41fc-9bb9-38b9ca2688be'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:13.951095 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-80d79e90-6fd8-41fc-9bb9-38b9ca2688be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3757af7375eb4492b5b0288438948837', 'request_id': 'req-80d79e90-6fd8-41fc-9bb9-38b9ca2688be'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:13.963831 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-80d79e90-6fd8-41fc-9bb9-38b9ca2688be admin admin] The token cache is being invalidate because project 3757af7375eb4492b5b0288438948837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:28:13.964133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-80d79e90-6fd8-41fc-9bb9-38b9ca2688be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:13.964991 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-80d79e90-6fd8-41fc-9bb9-38b9ca2688be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:13.967515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1070/2130] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:28:13 2022] DELETE /identity/v3/projects/3757af7375eb4492b5b0288438948837 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:13.972295 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5063f-82cb-48b6-b986-7a9ed7dfd846 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:13.972781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5063f-82cb-48b6-b986-7a9ed7dfd846 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:13.972876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5063f-82cb-48b6-b986-7a9ed7dfd846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:13.999826 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-52c5063f-82cb-48b6-b986-7a9ed7dfd846 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:14.001159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-52c5063f-82cb-48b6-b986-7a9ed7dfd846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:14.138944 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1061/2131] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:14.147527 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024e1f0-b33f-444d-aed2-7b23806b6045 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:14.148142 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024e1f0-b33f-444d-aed2-7b23806b6045 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:14.148621 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0024e1f0-b33f-444d-aed2-7b23806b6045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:14.183239 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-0024e1f0-b33f-444d-aed2-7b23806b6045 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:14.184484 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-0024e1f0-b33f-444d-aed2-7b23806b6045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:14.198400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1071/2132] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:14.204721 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5063f-82cb-48b6-b986-7a9ed7dfd846 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:14.214551 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5063f-82cb-48b6-b986-7a9ed7dfd846 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:14.219979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25c0ae-5042-4fff-ba1d-a0533c844680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:14.220900 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25c0ae-5042-4fff-ba1d-a0533c844680 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:14.221243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25c0ae-5042-4fff-ba1d-a0533c844680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:14.221582 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f25c0ae-5042-4fff-ba1d-a0533c844680 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:14.222665 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25c0ae-5042-4fff-ba1d-a0533c844680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:28:14.223417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25c0ae-5042-4fff-ba1d-a0533c844680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:14.224829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f25c0ae-5042-4fff-ba1d-a0533c844680 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:14.243409 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1062/2133] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:28:14.244356 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024e1f0-b33f-444d-aed2-7b23806b6045 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:14.254926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0024e1f0-b33f-444d-aed2-7b23806b6045 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:14.259951 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82381391-b57c-4ed0-a45f-40bad6d80f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:14.260809 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82381391-b57c-4ed0-a45f-40bad6d80f4f admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:14.260975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82381391-b57c-4ed0-a45f-40bad6d80f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:14.261286 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82381391-b57c-4ed0-a45f-40bad6d80f4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:14.261932 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82381391-b57c-4ed0-a45f-40bad6d80f4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:14.263845 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82381391-b57c-4ed0-a45f-40bad6d80f4f admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:14.304226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1072/2134] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:28:14 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:28:14.306598 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25c0ae-5042-4fff-ba1d-a0533c844680 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:14.319932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f25c0ae-5042-4fff-ba1d-a0533c844680 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:14.432639 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99658b4f-3363-46c4-a08a-5e3b657b3054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:14.432639 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99658b4f-3363-46c4-a08a-5e3b657b3054 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:14.432639 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99658b4f-3363-46c4-a08a-5e3b657b3054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:14.432639 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99658b4f-3363-46c4-a08a-5e3b657b3054 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:14.433160 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99658b4f-3363-46c4-a08a-5e3b657b3054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:14.434874 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99658b4f-3363-46c4-a08a-5e3b657b3054 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:14.458895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:14.465515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1063/2135] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:28:14 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:28:14.469565 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82381391-b57c-4ed0-a45f-40bad6d80f4f admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:14.484475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82381391-b57c-4ed0-a45f-40bad6d80f4f admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:14.494007 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793ff48-4990-4002-a02b-7148603cedc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:14.494780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793ff48-4990-4002-a02b-7148603cedc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:14.495128 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793ff48-4990-4002-a02b-7148603cedc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:14.495459 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9793ff48-4990-4002-a02b-7148603cedc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:14.496340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793ff48-4990-4002-a02b-7148603cedc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:14.497964 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9793ff48-4990-4002-a02b-7148603cedc2 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:14.512048 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1073/2136] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:14.514554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99658b4f-3363-46c4-a08a-5e3b657b3054 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:14.527175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99658b4f-3363-46c4-a08a-5e3b657b3054 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:14.535971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020aa2af-edb6-4ac4-9a62-392c018e3161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:14.536656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020aa2af-edb6-4ac4-9a62-392c018e3161 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:14.536964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020aa2af-edb6-4ac4-9a62-392c018e3161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:14.537200 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020aa2af-edb6-4ac4-9a62-392c018e3161 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:14.537858 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020aa2af-edb6-4ac4-9a62-392c018e3161 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:14.539522 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020aa2af-edb6-4ac4-9a62-392c018e3161 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:14.554887 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1064/2137] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:14.559252 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793ff48-4990-4002-a02b-7148603cedc2 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:14.574138 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9793ff48-4990-4002-a02b-7148603cedc2 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:14.581328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936d176-8a34-4d33-b555-6b77f7b4a8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:14.581773 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936d176-8a34-4d33-b555-6b77f7b4a8d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:14.582130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936d176-8a34-4d33-b555-6b77f7b4a8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:14.582457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936d176-8a34-4d33-b555-6b77f7b4a8d8 admin admin] PATH_INFO: `/v3/projects/f1d08bfd23884fc6a517aeea2e1512e0/users/29090adbbf1d49cf9fa24f96babc5043/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:14.600803 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936d176-8a34-4d33-b555-6b77f7b4a8d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1d08bfd23884fc6a517aeea2e1512e0, user_id=29090adbbf1d49cf9fa24f96babc5043, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:14.603608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936d176-8a34-4d33-b555-6b77f7b4a8d8 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:14.623418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1074/2138] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:28:14 2022] PUT /identity/v3/projects/f1d08bfd23884fc6a517aeea2e1512e0/users/29090adbbf1d49cf9fa24f96babc5043/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:14.701795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2bab7-02d1-43bb-acd9-00bb340e7409 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:14.702080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2bab7-02d1-43bb-acd9-00bb340e7409 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:14.702354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2bab7-02d1-43bb-acd9-00bb340e7409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:14.744157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-c2b2bab7-02d1-43bb-acd9-00bb340e7409 None None] MFA Rules not processed for user `29090adbbf1d49cf9fa24f96babc5043`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:14.745417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c2b2bab7-02d1-43bb-acd9-00bb340e7409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:14.837215 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1065/2139] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:14.847010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936d176-8a34-4d33-b555-6b77f7b4a8d8 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:14.849296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-7936d176-8a34-4d33-b555-6b77f7b4a8d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:14.876243 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:14.922552 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936d176-8a34-4d33-b555-6b77f7b4a8d8 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:14.926184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651dcd1c-2207-4752-935d-e3f455ddfdf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:14.926920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dcd1c-2207-4752-935d-e3f455ddfdf3 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:14.927165 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dcd1c-2207-4752-935d-e3f455ddfdf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:14.927432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651dcd1c-2207-4752-935d-e3f455ddfdf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:14.941899 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651dcd1c-2207-4752-935d-e3f455ddfdf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:14.943337 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651dcd1c-2207-4752-935d-e3f455ddfdf3 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:14.953263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1075/2140] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:28:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:28:14.979562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2bab7-02d1-43bb-acd9-00bb340e7409 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:15.043192 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2bab7-02d1-43bb-acd9-00bb340e7409 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:15.050773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7252294-e97d-48fb-b219-626157318d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:15.051423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7252294-e97d-48fb-b219-626157318d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:15.051705 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7252294-e97d-48fb-b219-626157318d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:15.052007 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7252294-e97d-48fb-b219-626157318d8f admin admin] PATH_INFO: `/v3/users/29090adbbf1d49cf9fa24f96babc5043` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:15.053259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7252294-e97d-48fb-b219-626157318d8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=29090adbbf1d49cf9fa24f96babc5043)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:15.055335 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7252294-e97d-48fb-b219-626157318d8f admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:15.105466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-f7252294-e97d-48fb-b219-626157318d8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29090adbbf1d49cf9fa24f96babc5043'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:15.113051 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-f7252294-e97d-48fb-b219-626157318d8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29090adbbf1d49cf9fa24f96babc5043'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:15.129146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-f7252294-e97d-48fb-b219-626157318d8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29090adbbf1d49cf9fa24f96babc5043'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:15.129931 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-f7252294-e97d-48fb-b219-626157318d8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29090adbbf1d49cf9fa24f96babc5043'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:15.140374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-f7252294-e97d-48fb-b219-626157318d8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29090adbbf1d49cf9fa24f96babc5043'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:15.142402 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:15.152369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1066/2141] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:28:14 2022] DELETE /identity/v3/users/29090adbbf1d49cf9fa24f96babc5043 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:15.157291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39e4ea-2a18-4a45-9e6d-13dd1c33d767 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:15.157820 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39e4ea-2a18-4a45-9e6d-13dd1c33d767 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:15.157970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39e4ea-2a18-4a45-9e6d-13dd1c33d767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:15.179458 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-6e39e4ea-2a18-4a45-9e6d-13dd1c33d767 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:15.180734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6e39e4ea-2a18-4a45-9e6d-13dd1c33d767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:15.249118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1076/2142] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:28:15.259446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7252294-e97d-48fb-b219-626157318d8f admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:15.261033 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f7252294-e97d-48fb-b219-626157318d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:15.324133 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7252294-e97d-48fb-b219-626157318d8f admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:15.327824 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552800d-3ea5-4980-90cf-f4c48fb85632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:15.328649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552800d-3ea5-4980-90cf-f4c48fb85632 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:15.328814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552800d-3ea5-4980-90cf-f4c48fb85632 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:15.329087 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552800d-3ea5-4980-90cf-f4c48fb85632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:15.339605 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552800d-3ea5-4980-90cf-f4c48fb85632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:15.341371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0552800d-3ea5-4980-90cf-f4c48fb85632 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:15.352168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1067/2143] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:28:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:28:15.636974 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39e4ea-2a18-4a45-9e6d-13dd1c33d767 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:15.638524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6e39e4ea-2a18-4a45-9e6d-13dd1c33d767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:15.651946 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39e4ea-2a18-4a45-9e6d-13dd1c33d767 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:15.656997 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:15.657671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:15.658006 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:15.658559 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b admin admin] PATH_INFO: `/v3/projects/f1d08bfd23884fc6a517aeea2e1512e0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:15.659789 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1d08bfd23884fc6a517aeea2e1512e0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:15.661702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:15.722206 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1d08bfd23884fc6a517aeea2e1512e0', 'request_id': 'req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:15.723981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1d08bfd23884fc6a517aeea2e1512e0', 'request_id': 'req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:15.732485 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1d08bfd23884fc6a517aeea2e1512e0', 'request_id': 'req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:15.744088 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b admin admin] The token cache is being invalidate because project f1d08bfd23884fc6a517aeea2e1512e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:28:15.744379 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:15.745029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:15.746552 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1077/2144] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:28:15 2022] DELETE /identity/v3/projects/f1d08bfd23884fc6a517aeea2e1512e0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:22.230313 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564749d-30ce-4a05-ac6d-b3df3babe0da None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:22.230812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564749d-30ce-4a05-ac6d-b3df3babe0da None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:22.230926 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564749d-30ce-4a05-ac6d-b3df3babe0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:22.251044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-d564749d-30ce-4a05-ac6d-b3df3babe0da None None] MFA Rules not processed for user `be3e2afc428a45e2b993c39e3d53cd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:22.252901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-d564749d-30ce-4a05-ac6d-b3df3babe0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:22.314117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1068/2145] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:28:25.741133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:32.306408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=52 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:38.150780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:38.152334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:38.231084 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5d860-80c0-4aa7-adc4-4b32d1d2d15b admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:38.235602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750058d9-d551-4aa4-9e1f-fa05189a0742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:38.236416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750058d9-d551-4aa4-9e1f-fa05189a0742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:38.236695 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750058d9-d551-4aa4-9e1f-fa05189a0742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:38.237095 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750058d9-d551-4aa4-9e1f-fa05189a0742 admin admin] PATH_INFO: `/v3/users/8021b97fadf4438ea002f49483461b38` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:38.238289 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750058d9-d551-4aa4-9e1f-fa05189a0742 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8021b97fadf4438ea002f49483461b38)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:38.240247 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750058d9-d551-4aa4-9e1f-fa05189a0742 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:38.289829 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-750058d9-d551-4aa4-9e1f-fa05189a0742 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8021b97fadf4438ea002f49483461b38'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:38.305127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-750058d9-d551-4aa4-9e1f-fa05189a0742 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8021b97fadf4438ea002f49483461b38'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:38.313176 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-750058d9-d551-4aa4-9e1f-fa05189a0742 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8021b97fadf4438ea002f49483461b38'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:38.314050 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-750058d9-d551-4aa4-9e1f-fa05189a0742 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8021b97fadf4438ea002f49483461b38'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:38.314765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-750058d9-d551-4aa4-9e1f-fa05189a0742 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8021b97fadf4438ea002f49483461b38'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:38.334243 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1078/2146] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:28:38 2022] DELETE /identity/v3/users/8021b97fadf4438ea002f49483461b38 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:38.338781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393ea36-cfab-4fb8-9561-f40f283d1c7e None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:38.339330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393ea36-cfab-4fb8-9561-f40f283d1c7e None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:38.339330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393ea36-cfab-4fb8-9561-f40f283d1c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:38.358728 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-4393ea36-cfab-4fb8-9561-f40f283d1c7e None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:38.359864 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4393ea36-cfab-4fb8-9561-f40f283d1c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:38.431243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1069/2147] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:38.437982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750058d9-d551-4aa4-9e1f-fa05189a0742 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:38.439648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-750058d9-d551-4aa4-9e1f-fa05189a0742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:38.514817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750058d9-d551-4aa4-9e1f-fa05189a0742 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:38.519196 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd74c5-4dc0-43c9-b316-76f25c4ab337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:38.519817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd74c5-4dc0-43c9-b316-76f25c4ab337 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:38.520211 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd74c5-4dc0-43c9-b316-76f25c4ab337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:38.520508 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd74c5-4dc0-43c9-b316-76f25c4ab337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:38.532176 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd74c5-4dc0-43c9-b316-76f25c4ab337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:38.533717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fd74c5-4dc0-43c9-b316-76f25c4ab337 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:38.544533 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1079/2148] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:28:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:28:38.845075 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393ea36-cfab-4fb8-9561-f40f283d1c7e None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:38.846757 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4393ea36-cfab-4fb8-9561-f40f283d1c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:38.862690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393ea36-cfab-4fb8-9561-f40f283d1c7e None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:38.869375 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7f683-8c2a-4086-8054-f0765251a5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:38.870066 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7f683-8c2a-4086-8054-f0765251a5ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:38.870649 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7f683-8c2a-4086-8054-f0765251a5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:38.870874 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7f683-8c2a-4086-8054-f0765251a5ae admin admin] PATH_INFO: `/v3/projects/ecd1d84a80424060940081d5b2b6bf23` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:38.872379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7f683-8c2a-4086-8054-f0765251a5ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecd1d84a80424060940081d5b2b6bf23)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:38.874474 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7f683-8c2a-4086-8054-f0765251a5ae admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:38.928015 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-20d7f683-8c2a-4086-8054-f0765251a5ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecd1d84a80424060940081d5b2b6bf23', 'request_id': 'req-20d7f683-8c2a-4086-8054-f0765251a5ae'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:38.934193 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-20d7f683-8c2a-4086-8054-f0765251a5ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecd1d84a80424060940081d5b2b6bf23', 'request_id': 'req-20d7f683-8c2a-4086-8054-f0765251a5ae'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:38.940576 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-20d7f683-8c2a-4086-8054-f0765251a5ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecd1d84a80424060940081d5b2b6bf23', 'request_id': 'req-20d7f683-8c2a-4086-8054-f0765251a5ae'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:38.957240 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-20d7f683-8c2a-4086-8054-f0765251a5ae admin admin] The token cache is being invalidate because project ecd1d84a80424060940081d5b2b6bf23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:28:38.957527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-20d7f683-8c2a-4086-8054-f0765251a5ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:38.958180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-20d7f683-8c2a-4086-8054-f0765251a5ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:38.959691 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1070/2149] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:28:38 2022] DELETE /identity/v3/projects/ecd1d84a80424060940081d5b2b6bf23 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:38.964249 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd74c5-4dc0-43c9-b316-76f25c4ab337 service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:38.974884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fd74c5-4dc0-43c9-b316-76f25c4ab337 service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:38.995293 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:39.043171 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381112ba-4073-4fbe-99da-7ad027db941a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:39.043823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381112ba-4073-4fbe-99da-7ad027db941a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:39.044138 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381112ba-4073-4fbe-99da-7ad027db941a admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:39.044431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381112ba-4073-4fbe-99da-7ad027db941a admin admin] PATH_INFO: `/v3/users/d78dc233e52c442a84f9bd684a79d47c` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:39.045698 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381112ba-4073-4fbe-99da-7ad027db941a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d78dc233e52c442a84f9bd684a79d47c)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:39.048096 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381112ba-4073-4fbe-99da-7ad027db941a admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:39.110387 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-381112ba-4073-4fbe-99da-7ad027db941a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd78dc233e52c442a84f9bd684a79d47c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:39.122694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-381112ba-4073-4fbe-99da-7ad027db941a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd78dc233e52c442a84f9bd684a79d47c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:39.133019 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-381112ba-4073-4fbe-99da-7ad027db941a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd78dc233e52c442a84f9bd684a79d47c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:39.133801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-381112ba-4073-4fbe-99da-7ad027db941a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd78dc233e52c442a84f9bd684a79d47c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:39.134512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-381112ba-4073-4fbe-99da-7ad027db941a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd78dc233e52c442a84f9bd684a79d47c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:39.154052 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1080/2150] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:28:38 2022] DELETE /identity/v3/users/d78dc233e52c442a84f9bd684a79d47c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:39.570579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7f683-8c2a-4086-8054-f0765251a5ae admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:39.573339 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-20d7f683-8c2a-4086-8054-f0765251a5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:39.589307 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7f683-8c2a-4086-8054-f0765251a5ae admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:39.655923 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03a6db-5f1a-4f40-8362-930476285806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:39.656726 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03a6db-5f1a-4f40-8362-930476285806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:39.657002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03a6db-5f1a-4f40-8362-930476285806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:39.657275 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03a6db-5f1a-4f40-8362-930476285806 admin admin] PATH_INFO: `/v3/projects/24c0f4f2507848b9947c5608ceb15bec` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:39.658663 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03a6db-5f1a-4f40-8362-930476285806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24c0f4f2507848b9947c5608ceb15bec)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:39.660667 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03a6db-5f1a-4f40-8362-930476285806 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:39.698069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:39.720710 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6b03a6db-5f1a-4f40-8362-930476285806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24c0f4f2507848b9947c5608ceb15bec', 'request_id': 'req-6b03a6db-5f1a-4f40-8362-930476285806'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:39.728017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6b03a6db-5f1a-4f40-8362-930476285806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24c0f4f2507848b9947c5608ceb15bec', 'request_id': 'req-6b03a6db-5f1a-4f40-8362-930476285806'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:39.735614 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6b03a6db-5f1a-4f40-8362-930476285806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24c0f4f2507848b9947c5608ceb15bec', 'request_id': 'req-6b03a6db-5f1a-4f40-8362-930476285806'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:39.756444 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6b03a6db-5f1a-4f40-8362-930476285806 admin admin] The token cache is being invalidate because project 24c0f4f2507848b9947c5608ceb15bec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:28:39.756936 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6b03a6db-5f1a-4f40-8362-930476285806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:39.757584 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6b03a6db-5f1a-4f40-8362-930476285806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:39.759397 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1071/2151] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:28:39 2022] DELETE /identity/v3/projects/24c0f4f2507848b9947c5608ceb15bec => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:39.764469 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c4532-a967-4291-9cb6-34cb71fdad9a None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:39.764861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c4532-a967-4291-9cb6-34cb71fdad9a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:39.765208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c4532-a967-4291-9cb6-34cb71fdad9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:39.796175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-cf3c4532-a967-4291-9cb6-34cb71fdad9a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:39.798309 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-cf3c4532-a967-4291-9cb6-34cb71fdad9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:39.881672 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1081/2152] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:39.889795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10227bb-4fd8-4f7e-a5b3-39fba17c83cf None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:39.890126 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10227bb-4fd8-4f7e-a5b3-39fba17c83cf None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:39.890416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10227bb-4fd8-4f7e-a5b3-39fba17c83cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:39.916800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-c10227bb-4fd8-4f7e-a5b3-39fba17c83cf None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:39.918372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c10227bb-4fd8-4f7e-a5b3-39fba17c83cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:39.932911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1072/2153] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:39.933971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c4532-a967-4291-9cb6-34cb71fdad9a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:39.944541 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c4532-a967-4291-9cb6-34cb71fdad9a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:39.949486 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da77ea8-7393-435a-b3e2-d4e72791dc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:39.950506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da77ea8-7393-435a-b3e2-d4e72791dc67 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:39.950790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da77ea8-7393-435a-b3e2-d4e72791dc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:39.951065 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da77ea8-7393-435a-b3e2-d4e72791dc67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:39.951728 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da77ea8-7393-435a-b3e2-d4e72791dc67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:28:39.952462 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da77ea8-7393-435a-b3e2-d4e72791dc67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:39.954480 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da77ea8-7393-435a-b3e2-d4e72791dc67 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:39.972100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1082/2154] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:28:39.976592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10227bb-4fd8-4f7e-a5b3-39fba17c83cf None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:39.988401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10227bb-4fd8-4f7e-a5b3-39fba17c83cf None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:39.993526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303861f-c6f7-4a28-b16d-fdf75dc92270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:39.994079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303861f-c6f7-4a28-b16d-fdf75dc92270 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:39.994368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303861f-c6f7-4a28-b16d-fdf75dc92270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:39.994626 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303861f-c6f7-4a28-b16d-fdf75dc92270 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:39.995255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303861f-c6f7-4a28-b16d-fdf75dc92270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:39.997207 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303861f-c6f7-4a28-b16d-fdf75dc92270 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:40.028668 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1073/2155] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:28:39 2022] POST /identity/v3/projects => generated 393 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:28:40.033643 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da77ea8-7393-435a-b3e2-d4e72791dc67 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:40.044348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da77ea8-7393-435a-b3e2-d4e72791dc67 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:40.073117 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:40.106386 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4c74e-288a-4497-93d9-656b9c480424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:40.106993 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4c74e-288a-4497-93d9-656b9c480424 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:40.107322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4c74e-288a-4497-93d9-656b9c480424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:40.107596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4c74e-288a-4497-93d9-656b9c480424 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:40.108274 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4c74e-288a-4497-93d9-656b9c480424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:40.110198 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4c74e-288a-4497-93d9-656b9c480424 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:40.135235 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1083/2156] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:28:40 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:28:40.139546 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303861f-c6f7-4a28-b16d-fdf75dc92270 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:40.150038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303861f-c6f7-4a28-b16d-fdf75dc92270 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:40.154731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b72e21-c3be-4219-9904-100260ad8521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:40.155338 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b72e21-c3be-4219-9904-100260ad8521 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:40.155617 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b72e21-c3be-4219-9904-100260ad8521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:40.155949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b72e21-c3be-4219-9904-100260ad8521 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:40.156562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b72e21-c3be-4219-9904-100260ad8521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:40.158301 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b72e21-c3be-4219-9904-100260ad8521 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:40.169063 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1074/2157] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:40.173718 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4c74e-288a-4497-93d9-656b9c480424 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:40.184833 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4c74e-288a-4497-93d9-656b9c480424 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:40.190491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e909206-66e4-4cc7-8d73-94732e96d159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:40.191119 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e909206-66e4-4cc7-8d73-94732e96d159 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:40.191400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e909206-66e4-4cc7-8d73-94732e96d159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:40.191667 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e909206-66e4-4cc7-8d73-94732e96d159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:40.192522 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e909206-66e4-4cc7-8d73-94732e96d159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:40.194195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e909206-66e4-4cc7-8d73-94732e96d159 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:40.204729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1084/2158] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:40.209124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b72e21-c3be-4219-9904-100260ad8521 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:40.219880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b72e21-c3be-4219-9904-100260ad8521 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:40.225145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431e1c0-6014-4367-9d36-f6a91f9bf498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:40.225751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431e1c0-6014-4367-9d36-f6a91f9bf498 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:40.226029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431e1c0-6014-4367-9d36-f6a91f9bf498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:40.226288 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431e1c0-6014-4367-9d36-f6a91f9bf498 admin admin] PATH_INFO: `/v3/projects/57d4fab7d8bc4481b529b21356381e92/users/65c93034b2f34275b3e049a6adae3acc/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:40.241604 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431e1c0-6014-4367-9d36-f6a91f9bf498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57d4fab7d8bc4481b529b21356381e92, user_id=65c93034b2f34275b3e049a6adae3acc, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:40.244379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e431e1c0-6014-4367-9d36-f6a91f9bf498 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:40.259009 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1075/2159] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:28:40 2022] PUT /identity/v3/projects/57d4fab7d8bc4481b529b21356381e92/users/65c93034b2f34275b3e049a6adae3acc/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:40.270544 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611a847-fe8d-4e36-96ee-32ee3a3da267 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:40.270808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611a847-fe8d-4e36-96ee-32ee3a3da267 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:40.271089 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611a847-fe8d-4e36-96ee-32ee3a3da267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:40.305921 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-0611a847-fe8d-4e36-96ee-32ee3a3da267 None None] MFA Rules not processed for user `65c93034b2f34275b3e049a6adae3acc`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:40.307192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-0611a847-fe8d-4e36-96ee-32ee3a3da267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:40.376272 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1085/2160] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:40.381267 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431e1c0-6014-4367-9d36-f6a91f9bf498 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:40.451619 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431e1c0-6014-4367-9d36-f6a91f9bf498 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:40.457543 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fc4f1-cb47-45db-affb-28da16d4f21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:40.458354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fc4f1-cb47-45db-affb-28da16d4f21e admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:40.458721 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fc4f1-cb47-45db-affb-28da16d4f21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:40.459058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2fc4f1-cb47-45db-affb-28da16d4f21e admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:40.459756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2fc4f1-cb47-45db-affb-28da16d4f21e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:40.461801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2fc4f1-cb47-45db-affb-28da16d4f21e admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:40.498199 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1076/2161] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:28:40 2022] POST /identity/v3/projects => generated 389 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:28:40.499347 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611a847-fe8d-4e36-96ee-32ee3a3da267 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:40.510713 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611a847-fe8d-4e36-96ee-32ee3a3da267 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:40.573231 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68bd22-d85f-4570-b55d-4d66215eff46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:40.573838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68bd22-d85f-4570-b55d-4d66215eff46 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:40.574173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68bd22-d85f-4570-b55d-4d66215eff46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:40.574382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a68bd22-d85f-4570-b55d-4d66215eff46 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:40.575015 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68bd22-d85f-4570-b55d-4d66215eff46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:40.576981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a68bd22-d85f-4570-b55d-4d66215eff46 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:40.606229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1086/2162] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:28:40 2022] POST /identity/v3/users => generated 301 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:28:40.615727 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fc4f1-cb47-45db-affb-28da16d4f21e admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:40.633452 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2fc4f1-cb47-45db-affb-28da16d4f21e admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:40.646141 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ee35f-fd51-4048-bf63-013de62980f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:40.646807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ee35f-fd51-4048-bf63-013de62980f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:40.647149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ee35f-fd51-4048-bf63-013de62980f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:40.647353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725ee35f-fd51-4048-bf63-013de62980f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:40.648045 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ee35f-fd51-4048-bf63-013de62980f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:40.653766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725ee35f-fd51-4048-bf63-013de62980f1 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:40.666185 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1077/2163] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:40.670760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68bd22-d85f-4570-b55d-4d66215eff46 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:40.681601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a68bd22-d85f-4570-b55d-4d66215eff46 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:40.686770 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464165fd-460e-43b3-b615-3e79ae5e716c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:40.687498 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464165fd-460e-43b3-b615-3e79ae5e716c admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:40.687778 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464165fd-460e-43b3-b615-3e79ae5e716c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:40.688072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464165fd-460e-43b3-b615-3e79ae5e716c admin admin] PATH_INFO: `/v3/projects/fee0e6aa2eb64d26b8c4900b4cb4eb50/users/50701e7ccec44d1a9523f87483d3afa2/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:40.705662 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464165fd-460e-43b3-b615-3e79ae5e716c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee0e6aa2eb64d26b8c4900b4cb4eb50, user_id=50701e7ccec44d1a9523f87483d3afa2, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:40.708385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464165fd-460e-43b3-b615-3e79ae5e716c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:40.723497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1087/2164] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:28:40 2022] PUT /identity/v3/projects/fee0e6aa2eb64d26b8c4900b4cb4eb50/users/50701e7ccec44d1a9523f87483d3afa2/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:40.727796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ee35f-fd51-4048-bf63-013de62980f1 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:40.738502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725ee35f-fd51-4048-bf63-013de62980f1 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:40.762210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:40.811464 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ce7240-77a6-4ef9-8e02-d12622d67d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:40.812125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce7240-77a6-4ef9-8e02-d12622d67d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:40.812404 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce7240-77a6-4ef9-8e02-d12622d67d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:40.812672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ce7240-77a6-4ef9-8e02-d12622d67d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:40.813314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce7240-77a6-4ef9-8e02-d12622d67d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:40.814997 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ce7240-77a6-4ef9-8e02-d12622d67d7a admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:40.825735 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1078/2165] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:40.830216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464165fd-460e-43b3-b615-3e79ae5e716c admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:40.836566 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:40.843839 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464165fd-460e-43b3-b615-3e79ae5e716c admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:40.849258 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefa23e-17bd-40c0-9f0c-443ada117206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:40.849871 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefa23e-17bd-40c0-9f0c-443ada117206 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:40.850272 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefa23e-17bd-40c0-9f0c-443ada117206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:40.850431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cefa23e-17bd-40c0-9f0c-443ada117206 admin admin] PATH_INFO: `/v3/projects/fee0e6aa2eb64d26b8c4900b4cb4eb50/users/50701e7ccec44d1a9523f87483d3afa2/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:40.852478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cefa23e-17bd-40c0-9f0c-443ada117206 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee0e6aa2eb64d26b8c4900b4cb4eb50, user_id=50701e7ccec44d1a9523f87483d3afa2, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:40.855194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cefa23e-17bd-40c0-9f0c-443ada117206 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:40.866851 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1088/2166] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:28:40 2022] PUT /identity/v3/projects/fee0e6aa2eb64d26b8c4900b4cb4eb50/users/50701e7ccec44d1a9523f87483d3afa2/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:40.877423 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71914f1e-e554-4d13-8b91-8a1aaf82ca4b None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:40.877711 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71914f1e-e554-4d13-8b91-8a1aaf82ca4b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:40.877983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71914f1e-e554-4d13-8b91-8a1aaf82ca4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:40.907249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-71914f1e-e554-4d13-8b91-8a1aaf82ca4b None None] MFA Rules not processed for user `50701e7ccec44d1a9523f87483d3afa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:40.908595 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-71914f1e-e554-4d13-8b91-8a1aaf82ca4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:40.992929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1079/2167] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:40 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:41.005100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefa23e-17bd-40c0-9f0c-443ada117206 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:41.007077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9cefa23e-17bd-40c0-9f0c-443ada117206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:41.092625 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cefa23e-17bd-40c0-9f0c-443ada117206 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:41.097005 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d3f53f-8bb8-4833-bce9-0907b16a0645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:41.097794 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3f53f-8bb8-4833-bce9-0907b16a0645 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:41.098203 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3f53f-8bb8-4833-bce9-0907b16a0645 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:41.098609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d3f53f-8bb8-4833-bce9-0907b16a0645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:41.108746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3f53f-8bb8-4833-bce9-0907b16a0645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:41.110253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d3f53f-8bb8-4833-bce9-0907b16a0645 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:41.120209 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1089/2168] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:28:41 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:28:41.169895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71914f1e-e554-4d13-8b91-8a1aaf82ca4b None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:41.171771 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-71914f1e-e554-4d13-8b91-8a1aaf82ca4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:41.185091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71914f1e-e554-4d13-8b91-8a1aaf82ca4b None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:41.188857 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a5beb-d94d-4a07-9de0-208df5070f48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:41.189506 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a5beb-d94d-4a07-9de0-208df5070f48 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:41.189785 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a5beb-d94d-4a07-9de0-208df5070f48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:41.190059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a5beb-d94d-4a07-9de0-208df5070f48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:41.250731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a5beb-d94d-4a07-9de0-208df5070f48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:41.254235 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a5beb-d94d-4a07-9de0-208df5070f48 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:41.264424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1080/2169] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:28:41 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:28:43.099928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a80a3-2505-4a89-b110-ab365c00dd08 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:43.100520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a80a3-2505-4a89-b110-ab365c00dd08 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:43.100520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a80a3-2505-4a89-b110-ab365c00dd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:43.154935 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-964a80a3-2505-4a89-b110-ab365c00dd08 None None] MFA Rules not processed for user `f643fd4ee6bc4ccfae45f3b8523ae46a`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:43.156636 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-964a80a3-2505-4a89-b110-ab365c00dd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:43.258871 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1090/2170] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:45.090270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a5beb-d94d-4a07-9de0-208df5070f48 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:45.149527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:45.173728 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a5beb-d94d-4a07-9de0-208df5070f48 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:45.178612 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457779d4-9797-4d15-906d-d8027ef2c611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:45.179393 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457779d4-9797-4d15-906d-d8027ef2c611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:45.179767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457779d4-9797-4d15-906d-d8027ef2c611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:45.180169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457779d4-9797-4d15-906d-d8027ef2c611 admin admin] PATH_INFO: `/v3/users/65c93034b2f34275b3e049a6adae3acc` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:45.181589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457779d4-9797-4d15-906d-d8027ef2c611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65c93034b2f34275b3e049a6adae3acc)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:45.183566 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457779d4-9797-4d15-906d-d8027ef2c611 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:45.242909 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-457779d4-9797-4d15-906d-d8027ef2c611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65c93034b2f34275b3e049a6adae3acc'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:45.251284 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-457779d4-9797-4d15-906d-d8027ef2c611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65c93034b2f34275b3e049a6adae3acc'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:45.269360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-457779d4-9797-4d15-906d-d8027ef2c611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65c93034b2f34275b3e049a6adae3acc'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:45.270251 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-457779d4-9797-4d15-906d-d8027ef2c611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65c93034b2f34275b3e049a6adae3acc'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:45.280240 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-457779d4-9797-4d15-906d-d8027ef2c611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65c93034b2f34275b3e049a6adae3acc'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:45.287741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1081/2171] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:28:45 2022] DELETE /identity/v3/users/65c93034b2f34275b3e049a6adae3acc => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:45.292553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1969dd-2d70-4ab6-9fbf-fe539430c76c None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:45.293077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1969dd-2d70-4ab6-9fbf-fe539430c76c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:45.293077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1969dd-2d70-4ab6-9fbf-fe539430c76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:45.349786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-9e1969dd-2d70-4ab6-9fbf-fe539430c76c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:45.351064 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9e1969dd-2d70-4ab6-9fbf-fe539430c76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:45.480043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1091/2172] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:45.487227 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457779d4-9797-4d15-906d-d8027ef2c611 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:45.488966 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-457779d4-9797-4d15-906d-d8027ef2c611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:45.552156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457779d4-9797-4d15-906d-d8027ef2c611 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:45.555809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab07de-1671-4c62-bae0-db5ef9202784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:45.556474 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab07de-1671-4c62-bae0-db5ef9202784 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:45.556754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab07de-1671-4c62-bae0-db5ef9202784 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:45.557022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab07de-1671-4c62-bae0-db5ef9202784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:45.568025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab07de-1671-4c62-bae0-db5ef9202784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:45.569523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab07de-1671-4c62-bae0-db5ef9202784 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:45.579658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1082/2173] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:28:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:28:45.883124 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1969dd-2d70-4ab6-9fbf-fe539430c76c None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:45.885847 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9e1969dd-2d70-4ab6-9fbf-fe539430c76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:45.906854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1969dd-2d70-4ab6-9fbf-fe539430c76c None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:45.919370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7bcd10-710b-4893-863b-ea15b4e929b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:45.920085 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bcd10-710b-4893-863b-ea15b4e929b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:45.920373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bcd10-710b-4893-863b-ea15b4e929b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:45.920626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bcd10-710b-4893-863b-ea15b4e929b4 admin admin] PATH_INFO: `/v3/projects/57d4fab7d8bc4481b529b21356381e92` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:45.922251 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7bcd10-710b-4893-863b-ea15b4e929b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57d4fab7d8bc4481b529b21356381e92)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:45.924624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7bcd10-710b-4893-863b-ea15b4e929b4 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:45.962271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:45.992508 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0d7bcd10-710b-4893-863b-ea15b4e929b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57d4fab7d8bc4481b529b21356381e92', 'request_id': 'req-0d7bcd10-710b-4893-863b-ea15b4e929b4'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:46.003846 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0d7bcd10-710b-4893-863b-ea15b4e929b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57d4fab7d8bc4481b529b21356381e92', 'request_id': 'req-0d7bcd10-710b-4893-863b-ea15b4e929b4'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:46.023957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0d7bcd10-710b-4893-863b-ea15b4e929b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57d4fab7d8bc4481b529b21356381e92', 'request_id': 'req-0d7bcd10-710b-4893-863b-ea15b4e929b4'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:46.053713 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0d7bcd10-710b-4893-863b-ea15b4e929b4 admin admin] The token cache is being invalidate because project 57d4fab7d8bc4481b529b21356381e92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:28:46.054008 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0d7bcd10-710b-4893-863b-ea15b4e929b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:46.055176 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0d7bcd10-710b-4893-863b-ea15b4e929b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:46.060277 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1092/2174] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:28:45 2022] DELETE /identity/v3/projects/57d4fab7d8bc4481b529b21356381e92 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:46.068808 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab07de-1671-4c62-bae0-db5ef9202784 service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:46.082381 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab07de-1671-4c62-bae0-db5ef9202784 service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:46.152111 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142abdf-df3f-47ed-8fb4-2adfd1175bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:46.153071 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142abdf-df3f-47ed-8fb4-2adfd1175bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:46.153512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142abdf-df3f-47ed-8fb4-2adfd1175bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:46.154085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142abdf-df3f-47ed-8fb4-2adfd1175bda admin admin] PATH_INFO: `/v3/users/50701e7ccec44d1a9523f87483d3afa2` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:46.155657 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142abdf-df3f-47ed-8fb4-2adfd1175bda admin admin] RBAC: Authorizing `identity:delete_user(user_id=50701e7ccec44d1a9523f87483d3afa2)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:46.158380 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142abdf-df3f-47ed-8fb4-2adfd1175bda admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:46.173310 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:46.224119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2142abdf-df3f-47ed-8fb4-2adfd1175bda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50701e7ccec44d1a9523f87483d3afa2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:46.234210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2142abdf-df3f-47ed-8fb4-2adfd1175bda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50701e7ccec44d1a9523f87483d3afa2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:46.254642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2142abdf-df3f-47ed-8fb4-2adfd1175bda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50701e7ccec44d1a9523f87483d3afa2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:46.255342 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2142abdf-df3f-47ed-8fb4-2adfd1175bda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50701e7ccec44d1a9523f87483d3afa2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:46.264138 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2142abdf-df3f-47ed-8fb4-2adfd1175bda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50701e7ccec44d1a9523f87483d3afa2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:46.271371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1083/2175] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:28:46 2022] DELETE /identity/v3/users/50701e7ccec44d1a9523f87483d3afa2 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:46.582367 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7bcd10-710b-4893-863b-ea15b4e929b4 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:46.584110 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-0d7bcd10-710b-4893-863b-ea15b4e929b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:46.609592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7bcd10-710b-4893-863b-ea15b4e929b4 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:46.693084 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4210c-fb3b-4055-a2a5-827bf160fb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:46.694147 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4210c-fb3b-4055-a2a5-827bf160fb5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:46.694365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4210c-fb3b-4055-a2a5-827bf160fb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:46.694640 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4210c-fb3b-4055-a2a5-827bf160fb5b admin admin] PATH_INFO: `/v3/projects/fee0e6aa2eb64d26b8c4900b4cb4eb50` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:46.695912 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4210c-fb3b-4055-a2a5-827bf160fb5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fee0e6aa2eb64d26b8c4900b4cb4eb50)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:46.708461 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe4210c-fb3b-4055-a2a5-827bf160fb5b admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:46.749063 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ffe4210c-fb3b-4055-a2a5-827bf160fb5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fee0e6aa2eb64d26b8c4900b4cb4eb50', 'request_id': 'req-ffe4210c-fb3b-4055-a2a5-827bf160fb5b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:46.755507 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ffe4210c-fb3b-4055-a2a5-827bf160fb5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fee0e6aa2eb64d26b8c4900b4cb4eb50', 'request_id': 'req-ffe4210c-fb3b-4055-a2a5-827bf160fb5b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:46.765597 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ffe4210c-fb3b-4055-a2a5-827bf160fb5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fee0e6aa2eb64d26b8c4900b4cb4eb50', 'request_id': 'req-ffe4210c-fb3b-4055-a2a5-827bf160fb5b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:46.775937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ffe4210c-fb3b-4055-a2a5-827bf160fb5b admin admin] The token cache is being invalidate because project fee0e6aa2eb64d26b8c4900b4cb4eb50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:28:46.776260 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ffe4210c-fb3b-4055-a2a5-827bf160fb5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:46.776876 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ffe4210c-fb3b-4055-a2a5-827bf160fb5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:46.778497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1093/2176] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:28:46 2022] DELETE /identity/v3/projects/fee0e6aa2eb64d26b8c4900b4cb4eb50 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:46.784321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602019e-53f5-407c-86c5-41bac0713b38 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:46.784611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602019e-53f5-407c-86c5-41bac0713b38 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:46.784900 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602019e-53f5-407c-86c5-41bac0713b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:46.805620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-3602019e-53f5-407c-86c5-41bac0713b38 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:46.806836 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3602019e-53f5-407c-86c5-41bac0713b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:46.868491 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1084/2177] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:28:46.876811 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2fa12-cd29-4abd-9454-9a35bdbf71a8 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:46.877104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2fa12-cd29-4abd-9454-9a35bdbf71a8 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:46.877395 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2fa12-cd29-4abd-9454-9a35bdbf71a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:46.897992 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-4df2fa12-cd29-4abd-9454-9a35bdbf71a8 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:46.899194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4df2fa12-cd29-4abd-9454-9a35bdbf71a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:46.908411 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1094/2178] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:46.913375 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602019e-53f5-407c-86c5-41bac0713b38 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:46.924615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602019e-53f5-407c-86c5-41bac0713b38 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:46.930766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02530db2-c882-46e3-ba3b-2168e0bbc8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:46.931381 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02530db2-c882-46e3-ba3b-2168e0bbc8c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:46.931659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02530db2-c882-46e3-ba3b-2168e0bbc8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:46.931930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02530db2-c882-46e3-ba3b-2168e0bbc8c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:46.932624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02530db2-c882-46e3-ba3b-2168e0bbc8c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:28:46.932924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02530db2-c882-46e3-ba3b-2168e0bbc8c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:46.934557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02530db2-c882-46e3-ba3b-2168e0bbc8c0 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:46.947213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1085/2179] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:28:46.951914 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2fa12-cd29-4abd-9454-9a35bdbf71a8 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:46.962976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2fa12-cd29-4abd-9454-9a35bdbf71a8 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:46.968544 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216a68e-2ff1-4566-b434-019e6f6ae3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:46.969163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216a68e-2ff1-4566-b434-019e6f6ae3c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:46.969447 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216a68e-2ff1-4566-b434-019e6f6ae3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:46.969719 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216a68e-2ff1-4566-b434-019e6f6ae3c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:46.970390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d216a68e-2ff1-4566-b434-019e6f6ae3c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:46.972443 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d216a68e-2ff1-4566-b434-019e6f6ae3c5 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:47.002956 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1095/2180] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:28:46 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:28:47.007639 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02530db2-c882-46e3-ba3b-2168e0bbc8c0 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:47.022997 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02530db2-c882-46e3-ba3b-2168e0bbc8c0 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:47.083721 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58401365-f9d2-4c53-9057-c582d800be42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:47.084376 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58401365-f9d2-4c53-9057-c582d800be42 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:47.084672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58401365-f9d2-4c53-9057-c582d800be42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:47.084940 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58401365-f9d2-4c53-9057-c582d800be42 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:47.085552 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58401365-f9d2-4c53-9057-c582d800be42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:47.087927 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58401365-f9d2-4c53-9057-c582d800be42 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:47.113694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1086/2181] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:28:47 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:28:47.117664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216a68e-2ff1-4566-b434-019e6f6ae3c5 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:47.128445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216a68e-2ff1-4566-b434-019e6f6ae3c5 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:47.133475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dcbf39-0ac0-4cc2-85b0-2e70edb75097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:47.134163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcbf39-0ac0-4cc2-85b0-2e70edb75097 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:47.134446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcbf39-0ac0-4cc2-85b0-2e70edb75097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:47.134738 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dcbf39-0ac0-4cc2-85b0-2e70edb75097 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:47.135393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dcbf39-0ac0-4cc2-85b0-2e70edb75097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:47.137030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dcbf39-0ac0-4cc2-85b0-2e70edb75097 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:47.150603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1096/2182] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:47.154738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58401365-f9d2-4c53-9057-c582d800be42 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:47.160332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:47.167421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58401365-f9d2-4c53-9057-c582d800be42 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:47.173960 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f2d69-5b1a-4377-9fce-fb1587ebfc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:47.174579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f2d69-5b1a-4377-9fce-fb1587ebfc6c admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:47.174888 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f2d69-5b1a-4377-9fce-fb1587ebfc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:47.175185 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f2d69-5b1a-4377-9fce-fb1587ebfc6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:47.175849 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f2d69-5b1a-4377-9fce-fb1587ebfc6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:47.177513 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f2d69-5b1a-4377-9fce-fb1587ebfc6c admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:47.189353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1087/2183] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:47.193721 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dcbf39-0ac0-4cc2-85b0-2e70edb75097 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:47.207666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dcbf39-0ac0-4cc2-85b0-2e70edb75097 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:47.214342 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad8261-9ebe-4098-9579-15d5cfc68905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:47.214663 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad8261-9ebe-4098-9579-15d5cfc68905 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:47.214839 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad8261-9ebe-4098-9579-15d5cfc68905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:47.215124 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad8261-9ebe-4098-9579-15d5cfc68905 admin admin] PATH_INFO: `/v3/projects/1519e4e00d204bc7b279b89bb6589dfc/users/310c3f29dde74a40960761de5d9f26c5/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:47.225882 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:47.234423 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad8261-9ebe-4098-9579-15d5cfc68905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1519e4e00d204bc7b279b89bb6589dfc, user_id=310c3f29dde74a40960761de5d9f26c5, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:47.237689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad8261-9ebe-4098-9579-15d5cfc68905 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:47.255734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1097/2184] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:28:47 2022] PUT /identity/v3/projects/1519e4e00d204bc7b279b89bb6589dfc/users/310c3f29dde74a40960761de5d9f26c5/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:47.266878 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bdf59-378a-41b6-92a2-7ffdc3a87ead None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:47.267166 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bdf59-378a-41b6-92a2-7ffdc3a87ead None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:47.267519 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bdf59-378a-41b6-92a2-7ffdc3a87ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:47.299512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-987bdf59-378a-41b6-92a2-7ffdc3a87ead None None] MFA Rules not processed for user `310c3f29dde74a40960761de5d9f26c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:47.302855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-987bdf59-378a-41b6-92a2-7ffdc3a87ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:47.368648 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1088/2185] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:28:47.373483 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad8261-9ebe-4098-9579-15d5cfc68905 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:47.452948 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad8261-9ebe-4098-9579-15d5cfc68905 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:47.458090 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad175f27-d377-4ef7-b5c0-bf516b4fc611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:47.459191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad175f27-d377-4ef7-b5c0-bf516b4fc611 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:47.459191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad175f27-d377-4ef7-b5c0-bf516b4fc611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:47.459370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad175f27-d377-4ef7-b5c0-bf516b4fc611 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:47.459909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad175f27-d377-4ef7-b5c0-bf516b4fc611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:47.461899 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad175f27-d377-4ef7-b5c0-bf516b4fc611 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:47.506884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1098/2186] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:28:47 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:28:47.511116 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987bdf59-378a-41b6-92a2-7ffdc3a87ead None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:47.523662 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987bdf59-378a-41b6-92a2-7ffdc3a87ead None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:47.617374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b26905-7bed-4b0d-b1f1-685f17222a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:47.618004 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b26905-7bed-4b0d-b1f1-685f17222a51 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:47.618297 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b26905-7bed-4b0d-b1f1-685f17222a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:47.618873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b26905-7bed-4b0d-b1f1-685f17222a51 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:47.620329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b26905-7bed-4b0d-b1f1-685f17222a51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:47.621500 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b26905-7bed-4b0d-b1f1-685f17222a51 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:47.644997 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1089/2187] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:28:47 2022] POST /identity/v3/users => generated 289 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:28:47.649212 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad175f27-d377-4ef7-b5c0-bf516b4fc611 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:47.660025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad175f27-d377-4ef7-b5c0-bf516b4fc611 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:47.664692 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef6e7d-9fa6-4d40-8baa-e7d445f5be1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:47.665312 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef6e7d-9fa6-4d40-8baa-e7d445f5be1f admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:47.665596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef6e7d-9fa6-4d40-8baa-e7d445f5be1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:47.665871 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ef6e7d-9fa6-4d40-8baa-e7d445f5be1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:47.666550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef6e7d-9fa6-4d40-8baa-e7d445f5be1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:47.668192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ef6e7d-9fa6-4d40-8baa-e7d445f5be1f admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:47.679744 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1099/2188] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:47.686755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b26905-7bed-4b0d-b1f1-685f17222a51 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:47.697422 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b26905-7bed-4b0d-b1f1-685f17222a51 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:47.702630 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac95686-4b7b-47f3-9857-ea4164694504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:47.703248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac95686-4b7b-47f3-9857-ea4164694504 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:47.703527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac95686-4b7b-47f3-9857-ea4164694504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:47.703794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac95686-4b7b-47f3-9857-ea4164694504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:47.704523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac95686-4b7b-47f3-9857-ea4164694504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:47.706188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac95686-4b7b-47f3-9857-ea4164694504 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:47.722245 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1090/2189] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:47.726728 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef6e7d-9fa6-4d40-8baa-e7d445f5be1f admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:47.737691 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ef6e7d-9fa6-4d40-8baa-e7d445f5be1f admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:47.742736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60b4dc-8476-41f2-bcf4-14daa8e41bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:47.743417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60b4dc-8476-41f2-bcf4-14daa8e41bcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:47.743692 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60b4dc-8476-41f2-bcf4-14daa8e41bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:47.743961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60b4dc-8476-41f2-bcf4-14daa8e41bcc admin admin] PATH_INFO: `/v3/projects/8d0de3728b924006ad5e92de85d4a7f7/users/b44b48870ec84173bbdfb8e0d2367c52/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:47.764645 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60b4dc-8476-41f2-bcf4-14daa8e41bcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d0de3728b924006ad5e92de85d4a7f7, user_id=b44b48870ec84173bbdfb8e0d2367c52, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:47.767369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60b4dc-8476-41f2-bcf4-14daa8e41bcc admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:47.793294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1100/2190] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:28:47 2022] PUT /identity/v3/projects/8d0de3728b924006ad5e92de85d4a7f7/users/b44b48870ec84173bbdfb8e0d2367c52/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:28:47.795296 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1046e7b-7151-42ab-8cda-2ceb098475e2 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:47.795818 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1046e7b-7151-42ab-8cda-2ceb098475e2 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:47.796161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1046e7b-7151-42ab-8cda-2ceb098475e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:47.836407 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-a1046e7b-7151-42ab-8cda-2ceb098475e2 None None] MFA Rules not processed for user `b44b48870ec84173bbdfb8e0d2367c52`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:47.837709 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a1046e7b-7151-42ab-8cda-2ceb098475e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:47.904805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:47.933448 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60b4dc-8476-41f2-bcf4-14daa8e41bcc admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:47.943719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1091/2191] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:48.030947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60b4dc-8476-41f2-bcf4-14daa8e41bcc admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:48.036090 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94784f1c-a450-458b-8cc1-9859dc17bb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:48.036814 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94784f1c-a450-458b-8cc1-9859dc17bb0b admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:48.037164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94784f1c-a450-458b-8cc1-9859dc17bb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:48.037692 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94784f1c-a450-458b-8cc1-9859dc17bb0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:48.038332 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94784f1c-a450-458b-8cc1-9859dc17bb0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:48.040296 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94784f1c-a450-458b-8cc1-9859dc17bb0b admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:48.074355 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1101/2192] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:28:47 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:28:48.078468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1046e7b-7151-42ab-8cda-2ceb098475e2 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:48.093252 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1046e7b-7151-42ab-8cda-2ceb098475e2 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:48.157981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b48cf6-84e1-43d8-ac4c-a54405fcb96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:48.158673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b48cf6-84e1-43d8-ac4c-a54405fcb96b admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:48.158764 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b48cf6-84e1-43d8-ac4c-a54405fcb96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:48.159016 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b48cf6-84e1-43d8-ac4c-a54405fcb96b admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:48.159659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b48cf6-84e1-43d8-ac4c-a54405fcb96b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:48.161639 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b48cf6-84e1-43d8-ac4c-a54405fcb96b admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:48.188919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1092/2193] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:28:48 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:28:48.190433 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94784f1c-a450-458b-8cc1-9859dc17bb0b admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:48.202097 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94784f1c-a450-458b-8cc1-9859dc17bb0b admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:48.207343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9947187-2e85-45d5-80b0-4a8c530c92c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:48.207913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9947187-2e85-45d5-80b0-4a8c530c92c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:48.208274 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9947187-2e85-45d5-80b0-4a8c530c92c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:48.208548 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9947187-2e85-45d5-80b0-4a8c530c92c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:48.209203 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9947187-2e85-45d5-80b0-4a8c530c92c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:48.210901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9947187-2e85-45d5-80b0-4a8c530c92c6 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:48.224083 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1102/2194] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:48.228428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b48cf6-84e1-43d8-ac4c-a54405fcb96b admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:48.238801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b48cf6-84e1-43d8-ac4c-a54405fcb96b admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:48.243591 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55901a46-428e-4213-9df7-6ddf394f3bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:48.244288 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55901a46-428e-4213-9df7-6ddf394f3bed admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:48.244562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55901a46-428e-4213-9df7-6ddf394f3bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:48.244854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55901a46-428e-4213-9df7-6ddf394f3bed admin admin] PATH_INFO: `/v3/projects/bce7a90964c340b2bc81964f20124f4e/users/eb45dce1e5a24afb8d1eb10b1b60621f/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:48.262868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55901a46-428e-4213-9df7-6ddf394f3bed admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce7a90964c340b2bc81964f20124f4e, user_id=eb45dce1e5a24afb8d1eb10b1b60621f, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:48.265808 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55901a46-428e-4213-9df7-6ddf394f3bed admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:48.281128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1093/2195] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:28:48 2022] PUT /identity/v3/projects/bce7a90964c340b2bc81964f20124f4e/users/eb45dce1e5a24afb8d1eb10b1b60621f/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:48.285279 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9947187-2e85-45d5-80b0-4a8c530c92c6 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:48.296085 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9947187-2e85-45d5-80b0-4a8c530c92c6 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:48.306699 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:48.361824 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8cf82-d1b6-412b-8dd0-0ab5ee720d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:48.362553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8cf82-d1b6-412b-8dd0-0ab5ee720d43 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:48.362830 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8cf82-d1b6-412b-8dd0-0ab5ee720d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:48.363144 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8cf82-d1b6-412b-8dd0-0ab5ee720d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:48.363851 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8cf82-d1b6-412b-8dd0-0ab5ee720d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:48.365583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8cf82-d1b6-412b-8dd0-0ab5ee720d43 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:48.380314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1103/2196] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:48.384551 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55901a46-428e-4213-9df7-6ddf394f3bed admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:48.395122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55901a46-428e-4213-9df7-6ddf394f3bed admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:48.401479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5b1ec-ae08-4cf5-9d2b-531d722dc6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:48.402330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5b1ec-ae08-4cf5-9d2b-531d722dc6d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:48.403180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5b1ec-ae08-4cf5-9d2b-531d722dc6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:48.403180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5b1ec-ae08-4cf5-9d2b-531d722dc6d9 admin admin] PATH_INFO: `/v3/projects/bce7a90964c340b2bc81964f20124f4e/users/eb45dce1e5a24afb8d1eb10b1b60621f/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:48.405095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb5b1ec-ae08-4cf5-9d2b-531d722dc6d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce7a90964c340b2bc81964f20124f4e, user_id=eb45dce1e5a24afb8d1eb10b1b60621f, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:48.408260 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb5b1ec-ae08-4cf5-9d2b-531d722dc6d9 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:48.420567 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1094/2197] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:28:48 2022] PUT /identity/v3/projects/bce7a90964c340b2bc81964f20124f4e/users/eb45dce1e5a24afb8d1eb10b1b60621f/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:28:48.433037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11d1f9-ceb0-4340-8120-702320a17d60 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:48.433324 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11d1f9-ceb0-4340-8120-702320a17d60 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:48.433598 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11d1f9-ceb0-4340-8120-702320a17d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:48.463803 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-ee11d1f9-ceb0-4340-8120-702320a17d60 None None] MFA Rules not processed for user `eb45dce1e5a24afb8d1eb10b1b60621f`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:48.465144 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ee11d1f9-ceb0-4340-8120-702320a17d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:48.537593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1104/2198] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:48.549750 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5b1ec-ae08-4cf5-9d2b-531d722dc6d9 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:48.551670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9cb5b1ec-ae08-4cf5-9d2b-531d722dc6d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:48.619906 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5b1ec-ae08-4cf5-9d2b-531d722dc6d9 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:48.624220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721dca5d-de80-47b5-adc6-c5aee90fabf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:48.624847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721dca5d-de80-47b5-adc6-c5aee90fabf8 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:48.625128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721dca5d-de80-47b5-adc6-c5aee90fabf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:48.625401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721dca5d-de80-47b5-adc6-c5aee90fabf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:48.704133 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:48.707557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721dca5d-de80-47b5-adc6-c5aee90fabf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:48.709223 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721dca5d-de80-47b5-adc6-c5aee90fabf8 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:48.722597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1095/2199] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:28:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:28:50.367476 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11d1f9-ceb0-4340-8120-702320a17d60 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:50.377396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11d1f9-ceb0-4340-8120-702320a17d60 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:50.381187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccead66-8e64-4549-94c3-c6b4ed1bde95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:50.381828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccead66-8e64-4549-94c3-c6b4ed1bde95 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:50.382146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccead66-8e64-4549-94c3-c6b4ed1bde95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:50.382390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccead66-8e64-4549-94c3-c6b4ed1bde95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:50.443806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccead66-8e64-4549-94c3-c6b4ed1bde95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:50.445299 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccead66-8e64-4549-94c3-c6b4ed1bde95 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:50.454626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1105/2200] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:28:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:28:51.089820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721dca5d-de80-47b5-adc6-c5aee90fabf8 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:51.100509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721dca5d-de80-47b5-adc6-c5aee90fabf8 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:51.105337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7810eb0f-e489-4f4b-96ed-14fb0021598b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:51.105962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810eb0f-e489-4f4b-96ed-14fb0021598b service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:51.106381 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810eb0f-e489-4f4b-96ed-14fb0021598b service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:51.106658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7810eb0f-e489-4f4b-96ed-14fb0021598b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:51.118125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810eb0f-e489-4f4b-96ed-14fb0021598b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:51.120106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7810eb0f-e489-4f4b-96ed-14fb0021598b service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:51.131456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1096/2201] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:28:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:28:56.219334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccead66-8e64-4549-94c3-c6b4ed1bde95 service cinder] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:56.255728 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:56.301106 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccead66-8e64-4549-94c3-c6b4ed1bde95 service cinder] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:56.306308 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38969c-e423-4fab-8032-30ede6a68357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:56.307144 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38969c-e423-4fab-8032-30ede6a68357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:56.307397 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38969c-e423-4fab-8032-30ede6a68357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:56.307734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38969c-e423-4fab-8032-30ede6a68357 admin admin] PATH_INFO: `/v3/users/310c3f29dde74a40960761de5d9f26c5` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:56.308981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38969c-e423-4fab-8032-30ede6a68357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=310c3f29dde74a40960761de5d9f26c5)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:56.310970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38969c-e423-4fab-8032-30ede6a68357 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:56.365263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-8a38969c-e423-4fab-8032-30ede6a68357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '310c3f29dde74a40960761de5d9f26c5'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:56.378739 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-8a38969c-e423-4fab-8032-30ede6a68357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '310c3f29dde74a40960761de5d9f26c5'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:56.385988 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-8a38969c-e423-4fab-8032-30ede6a68357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '310c3f29dde74a40960761de5d9f26c5'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:56.386723 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-8a38969c-e423-4fab-8032-30ede6a68357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '310c3f29dde74a40960761de5d9f26c5'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:56.387330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-8a38969c-e423-4fab-8032-30ede6a68357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '310c3f29dde74a40960761de5d9f26c5'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:56.412958 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1106/2202] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:28:56 2022] DELETE /identity/v3/users/310c3f29dde74a40960761de5d9f26c5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:56.417791 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99eb46b-c15a-4621-919a-382248d835c6 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:56.418305 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99eb46b-c15a-4621-919a-382248d835c6 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:56.418435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99eb46b-c15a-4621-919a-382248d835c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:56.438456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-c99eb46b-c15a-4621-919a-382248d835c6 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:56.439713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c99eb46b-c15a-4621-919a-382248d835c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:56.504065 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1097/2203] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:56.514343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38969c-e423-4fab-8032-30ede6a68357 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:56.515868 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8a38969c-e423-4fab-8032-30ede6a68357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:56.581163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38969c-e423-4fab-8032-30ede6a68357 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:56.584815 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871d6a8-c176-48e0-aea1-301f9a1e4148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:56.585428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871d6a8-c176-48e0-aea1-301f9a1e4148 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:56.585706 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871d6a8-c176-48e0-aea1-301f9a1e4148 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:56.586009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0871d6a8-c176-48e0-aea1-301f9a1e4148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:56.595271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0871d6a8-c176-48e0-aea1-301f9a1e4148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:56.596803 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0871d6a8-c176-48e0-aea1-301f9a1e4148 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:56.606750 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1107/2204] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:28:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:28:56.900751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99eb46b-c15a-4621-919a-382248d835c6 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:56.903004 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c99eb46b-c15a-4621-919a-382248d835c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:56.918531 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99eb46b-c15a-4621-919a-382248d835c6 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:56.923677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca3d03-b65c-4cb0-814d-11d815d9d295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:56.924356 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca3d03-b65c-4cb0-814d-11d815d9d295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:56.924847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca3d03-b65c-4cb0-814d-11d815d9d295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:56.925127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca3d03-b65c-4cb0-814d-11d815d9d295 admin admin] PATH_INFO: `/v3/projects/1519e4e00d204bc7b279b89bb6589dfc` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:56.926283 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca3d03-b65c-4cb0-814d-11d815d9d295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1519e4e00d204bc7b279b89bb6589dfc)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:56.928253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca3d03-b65c-4cb0-814d-11d815d9d295 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:56.979467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-37ca3d03-b65c-4cb0-814d-11d815d9d295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1519e4e00d204bc7b279b89bb6589dfc', 'request_id': 'req-37ca3d03-b65c-4cb0-814d-11d815d9d295'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:56.985244 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-37ca3d03-b65c-4cb0-814d-11d815d9d295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1519e4e00d204bc7b279b89bb6589dfc', 'request_id': 'req-37ca3d03-b65c-4cb0-814d-11d815d9d295'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:56.991712 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-37ca3d03-b65c-4cb0-814d-11d815d9d295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1519e4e00d204bc7b279b89bb6589dfc', 'request_id': 'req-37ca3d03-b65c-4cb0-814d-11d815d9d295'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.004715 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-37ca3d03-b65c-4cb0-814d-11d815d9d295 admin admin] The token cache is being invalidate because project 1519e4e00d204bc7b279b89bb6589dfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:28:57.005189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-37ca3d03-b65c-4cb0-814d-11d815d9d295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.005815 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-37ca3d03-b65c-4cb0-814d-11d815d9d295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.007354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1098/2205] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:28:56 2022] DELETE /identity/v3/projects/1519e4e00d204bc7b279b89bb6589dfc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:57.011763 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871d6a8-c176-48e0-aea1-301f9a1e4148 service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:57.022111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0871d6a8-c176-48e0-aea1-301f9a1e4148 service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:57.072728 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:57.083720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75ae9a-4937-4871-bc50-063d9c35cb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:57.084431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75ae9a-4937-4871-bc50-063d9c35cb3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:57.084716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75ae9a-4937-4871-bc50-063d9c35cb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:57.085147 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd75ae9a-4937-4871-bc50-063d9c35cb3e admin admin] PATH_INFO: `/v3/users/b44b48870ec84173bbdfb8e0d2367c52` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:57.086770 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd75ae9a-4937-4871-bc50-063d9c35cb3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b44b48870ec84173bbdfb8e0d2367c52)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:57.089016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd75ae9a-4937-4871-bc50-063d9c35cb3e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:57.148114 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-dd75ae9a-4937-4871-bc50-063d9c35cb3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b44b48870ec84173bbdfb8e0d2367c52'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.161539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-dd75ae9a-4937-4871-bc50-063d9c35cb3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b44b48870ec84173bbdfb8e0d2367c52'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.172271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-dd75ae9a-4937-4871-bc50-063d9c35cb3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b44b48870ec84173bbdfb8e0d2367c52'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.173508 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-dd75ae9a-4937-4871-bc50-063d9c35cb3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b44b48870ec84173bbdfb8e0d2367c52'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.174904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-dd75ae9a-4937-4871-bc50-063d9c35cb3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b44b48870ec84173bbdfb8e0d2367c52'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.197376 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1108/2206] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:28:57 2022] DELETE /identity/v3/users/b44b48870ec84173bbdfb8e0d2367c52 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:57.531567 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca3d03-b65c-4cb0-814d-11d815d9d295 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:57.533283 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-37ca3d03-b65c-4cb0-814d-11d815d9d295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:57.556609 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca3d03-b65c-4cb0-814d-11d815d9d295 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:57.655443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f034ccc-665f-417e-9182-632d11d688b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:57.656307 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f034ccc-665f-417e-9182-632d11d688b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:57.656686 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f034ccc-665f-417e-9182-632d11d688b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:57.657184 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f034ccc-665f-417e-9182-632d11d688b7 admin admin] PATH_INFO: `/v3/projects/8d0de3728b924006ad5e92de85d4a7f7` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:57.658836 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f034ccc-665f-417e-9182-632d11d688b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d0de3728b924006ad5e92de85d4a7f7)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:57.661342 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f034ccc-665f-417e-9182-632d11d688b7 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:57.672042 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:57.730528 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3f034ccc-665f-417e-9182-632d11d688b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d0de3728b924006ad5e92de85d4a7f7', 'request_id': 'req-3f034ccc-665f-417e-9182-632d11d688b7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.738359 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3f034ccc-665f-417e-9182-632d11d688b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d0de3728b924006ad5e92de85d4a7f7', 'request_id': 'req-3f034ccc-665f-417e-9182-632d11d688b7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.746225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3f034ccc-665f-417e-9182-632d11d688b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d0de3728b924006ad5e92de85d4a7f7', 'request_id': 'req-3f034ccc-665f-417e-9182-632d11d688b7'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.764468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3f034ccc-665f-417e-9182-632d11d688b7 admin admin] The token cache is being invalidate because project 8d0de3728b924006ad5e92de85d4a7f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:28:57.764768 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3f034ccc-665f-417e-9182-632d11d688b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.765615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3f034ccc-665f-417e-9182-632d11d688b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.768278 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1099/2207] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:28:57 2022] DELETE /identity/v3/projects/8d0de3728b924006ad5e92de85d4a7f7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:57.771684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75ae9a-4937-4871-bc50-063d9c35cb3e admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:57.773178 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-dd75ae9a-4937-4871-bc50-063d9c35cb3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:57.787589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd75ae9a-4937-4871-bc50-063d9c35cb3e admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:57.850497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b76e83-fc6f-45e9-acf4-59dfce57b638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:57.851188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b76e83-fc6f-45e9-acf4-59dfce57b638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:57.851479 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b76e83-fc6f-45e9-acf4-59dfce57b638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:57.851748 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b76e83-fc6f-45e9-acf4-59dfce57b638 admin admin] PATH_INFO: `/v3/users/eb45dce1e5a24afb8d1eb10b1b60621f` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:57.852930 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b76e83-fc6f-45e9-acf4-59dfce57b638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb45dce1e5a24afb8d1eb10b1b60621f)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:57.855023 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b76e83-fc6f-45e9-acf4-59dfce57b638 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:57.905099 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-c8b76e83-fc6f-45e9-acf4-59dfce57b638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb45dce1e5a24afb8d1eb10b1b60621f'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.918359 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-c8b76e83-fc6f-45e9-acf4-59dfce57b638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb45dce1e5a24afb8d1eb10b1b60621f'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.926003 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-c8b76e83-fc6f-45e9-acf4-59dfce57b638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb45dce1e5a24afb8d1eb10b1b60621f'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.926748 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-c8b76e83-fc6f-45e9-acf4-59dfce57b638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb45dce1e5a24afb8d1eb10b1b60621f'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.927364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-c8b76e83-fc6f-45e9-acf4-59dfce57b638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb45dce1e5a24afb8d1eb10b1b60621f'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:57.929088 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:57.943856 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1109/2208] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:28:57 2022] DELETE /identity/v3/users/eb45dce1e5a24afb8d1eb10b1b60621f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:58.272732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f034ccc-665f-417e-9182-632d11d688b7 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:58.274215 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3f034ccc-665f-417e-9182-632d11d688b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:58.287939 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f034ccc-665f-417e-9182-632d11d688b7 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:58.357462 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250bf22-0e2b-470e-84e2-32aa730d4113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:58.358290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250bf22-0e2b-470e-84e2-32aa730d4113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:58.358672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250bf22-0e2b-470e-84e2-32aa730d4113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:58.359047 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250bf22-0e2b-470e-84e2-32aa730d4113 admin admin] PATH_INFO: `/v3/projects/bce7a90964c340b2bc81964f20124f4e` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:58.360392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250bf22-0e2b-470e-84e2-32aa730d4113 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bce7a90964c340b2bc81964f20124f4e)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:58.362550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250bf22-0e2b-470e-84e2-32aa730d4113 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:58.419087 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7250bf22-0e2b-470e-84e2-32aa730d4113 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bce7a90964c340b2bc81964f20124f4e', 'request_id': 'req-7250bf22-0e2b-470e-84e2-32aa730d4113'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:58.425825 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7250bf22-0e2b-470e-84e2-32aa730d4113 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bce7a90964c340b2bc81964f20124f4e', 'request_id': 'req-7250bf22-0e2b-470e-84e2-32aa730d4113'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:58.433584 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7250bf22-0e2b-470e-84e2-32aa730d4113 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bce7a90964c340b2bc81964f20124f4e', 'request_id': 'req-7250bf22-0e2b-470e-84e2-32aa730d4113'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:58.448750 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7250bf22-0e2b-470e-84e2-32aa730d4113 admin admin] The token cache is being invalidate because project bce7a90964c340b2bc81964f20124f4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:28:58.449043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7250bf22-0e2b-470e-84e2-32aa730d4113 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:58.449700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7250bf22-0e2b-470e-84e2-32aa730d4113 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:28:58.452219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1100/2209] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:28:58 2022] DELETE /identity/v3/projects/bce7a90964c340b2bc81964f20124f4e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:58.457925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82067e-151a-40d9-bec8-96869adbc91f None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:58.458348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82067e-151a-40d9-bec8-96869adbc91f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:58.458730 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82067e-151a-40d9-bec8-96869adbc91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:58.485673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-ea82067e-151a-40d9-bec8-96869adbc91f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:58.488667 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ea82067e-151a-40d9-bec8-96869adbc91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:58.561650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1110/2210] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:28:58.570070 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1028b5-c539-43ac-9193-6fea436602e5 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:58.570885 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1028b5-c539-43ac-9193-6fea436602e5 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:58.570885 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1028b5-c539-43ac-9193-6fea436602e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:58.591689 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-fb1028b5-c539-43ac-9193-6fea436602e5 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:58.592866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-fb1028b5-c539-43ac-9193-6fea436602e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:58.601201 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1101/2211] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:58.605624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82067e-151a-40d9-bec8-96869adbc91f None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:58.616054 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82067e-151a-40d9-bec8-96869adbc91f None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:58.620894 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a068e92-6d60-4007-9522-618fdd44b91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:58.621516 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a068e92-6d60-4007-9522-618fdd44b91f admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:58.621785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a068e92-6d60-4007-9522-618fdd44b91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:58.622053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a068e92-6d60-4007-9522-618fdd44b91f admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:58.622702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a068e92-6d60-4007-9522-618fdd44b91f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:28:58.623000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a068e92-6d60-4007-9522-618fdd44b91f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:58.624641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a068e92-6d60-4007-9522-618fdd44b91f admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:58.636926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1111/2212] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:28:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:28:58.641308 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1028b5-c539-43ac-9193-6fea436602e5 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:58.651956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1028b5-c539-43ac-9193-6fea436602e5 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:58.657693 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634fc52f-883e-4f3b-8e7f-b9bbf98efc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:58.658318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fc52f-883e-4f3b-8e7f-b9bbf98efc08 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:58.658601 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fc52f-883e-4f3b-8e7f-b9bbf98efc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:58.658893 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fc52f-883e-4f3b-8e7f-b9bbf98efc08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:58.659516 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634fc52f-883e-4f3b-8e7f-b9bbf98efc08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:58.661483 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634fc52f-883e-4f3b-8e7f-b9bbf98efc08 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:58.690274 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:58.710876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1102/2213] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:28:58 2022] POST /identity/v3/projects => generated 375 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:28:58.720922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a068e92-6d60-4007-9522-618fdd44b91f admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:58.737182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a068e92-6d60-4007-9522-618fdd44b91f admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:58.798714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21c357-7ea1-4113-a3fd-84a7cf8ddadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:58.799423 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21c357-7ea1-4113-a3fd-84a7cf8ddadf admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:58.799707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21c357-7ea1-4113-a3fd-84a7cf8ddadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:58.799983 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21c357-7ea1-4113-a3fd-84a7cf8ddadf admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:58.800648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21c357-7ea1-4113-a3fd-84a7cf8ddadf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:58.802676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21c357-7ea1-4113-a3fd-84a7cf8ddadf admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:58.837891 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1112/2214] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:28:58 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:28:58.839156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634fc52f-883e-4f3b-8e7f-b9bbf98efc08 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:58.854899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634fc52f-883e-4f3b-8e7f-b9bbf98efc08 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:58.860611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1548d7-378a-4e34-a4fb-1a3da1f11427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:58.861538 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1548d7-378a-4e34-a4fb-1a3da1f11427 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:58.862049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1548d7-378a-4e34-a4fb-1a3da1f11427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:58.862443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1548d7-378a-4e34-a4fb-1a3da1f11427 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:58.863319 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1548d7-378a-4e34-a4fb-1a3da1f11427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:58.865203 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1548d7-378a-4e34-a4fb-1a3da1f11427 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:58.880065 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1103/2215] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:58.884404 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21c357-7ea1-4113-a3fd-84a7cf8ddadf admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:58.892957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21c357-7ea1-4113-a3fd-84a7cf8ddadf admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:58.897992 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77667c70-260c-4d08-aa13-46959cd4c92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:58.898616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77667c70-260c-4d08-aa13-46959cd4c92b admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:58.898888 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77667c70-260c-4d08-aa13-46959cd4c92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:58.899276 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77667c70-260c-4d08-aa13-46959cd4c92b admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:58.899855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77667c70-260c-4d08-aa13-46959cd4c92b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:58.901451 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77667c70-260c-4d08-aa13-46959cd4c92b admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:58.916104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1113/2216] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:28:58.918757 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1548d7-378a-4e34-a4fb-1a3da1f11427 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:58.931379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1548d7-378a-4e34-a4fb-1a3da1f11427 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:58.949287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbafad-88f6-4594-82cd-5ac3a1ed0d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:58.950012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbafad-88f6-4594-82cd-5ac3a1ed0d38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:58.950371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbafad-88f6-4594-82cd-5ac3a1ed0d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:58.950557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbafad-88f6-4594-82cd-5ac3a1ed0d38 admin admin] PATH_INFO: `/v3/projects/cfb89d8a051a4da7850fa596d63c952b/users/165e5f6667ae42af9e87fde5d17ccc14/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:58.966343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbafad-88f6-4594-82cd-5ac3a1ed0d38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfb89d8a051a4da7850fa596d63c952b, user_id=165e5f6667ae42af9e87fde5d17ccc14, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:58.969272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbafad-88f6-4594-82cd-5ac3a1ed0d38 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:58.985173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1104/2217] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:28:58 2022] PUT /identity/v3/projects/cfb89d8a051a4da7850fa596d63c952b/users/165e5f6667ae42af9e87fde5d17ccc14/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:28:58.996979 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb767f4-f158-4ef1-8b7c-9732df798dc3 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:58.997310 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb767f4-f158-4ef1-8b7c-9732df798dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:58.997760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb767f4-f158-4ef1-8b7c-9732df798dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:59.044869 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-4bb767f4-f158-4ef1-8b7c-9732df798dc3 None None] MFA Rules not processed for user `165e5f6667ae42af9e87fde5d17ccc14`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:28:59.046736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4bb767f4-f158-4ef1-8b7c-9732df798dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:59.097760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:28:59.124403 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1114/2218] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:28:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:28:59.136027 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbafad-88f6-4594-82cd-5ac3a1ed0d38 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:28:59.137835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-60dbafad-88f6-4594-82cd-5ac3a1ed0d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:28:59.213638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbafad-88f6-4594-82cd-5ac3a1ed0d38 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:28:59.220271 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c4ae9-7be6-40a1-9e65-05554953e994 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:28:59.220271 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c4ae9-7be6-40a1-9e65-05554953e994 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:28:59.220271 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c4ae9-7be6-40a1-9e65-05554953e994 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:28:59.220271 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c4ae9-7be6-40a1-9e65-05554953e994 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:28:59.232462 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c4ae9-7be6-40a1-9e65-05554953e994 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:28:59.234050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c4ae9-7be6-40a1-9e65-05554953e994 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:28:59.246159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1105/2219] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:28:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:29:01.559421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb767f4-f158-4ef1-8b7c-9732df798dc3 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:01.561172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4bb767f4-f158-4ef1-8b7c-9732df798dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:01.634038 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb767f4-f158-4ef1-8b7c-9732df798dc3 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:01.637764 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a605113-3585-4864-bb09-2a02d6ffb69d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:01.638386 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a605113-3585-4864-bb09-2a02d6ffb69d service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:01.638752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a605113-3585-4864-bb09-2a02d6ffb69d service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:01.638944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a605113-3585-4864-bb09-2a02d6ffb69d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:01.640866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9a605113-3585-4864-bb09-2a02d6ffb69d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:01.724298 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a605113-3585-4864-bb09-2a02d6ffb69d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:01.725757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a605113-3585-4864-bb09-2a02d6ffb69d service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:01.738229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1115/2220] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:29:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:29:02.422767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c4ae9-7be6-40a1-9e65-05554953e994 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:02.426122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-ab6c4ae9-7be6-40a1-9e65-05554953e994 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:02.512821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c4ae9-7be6-40a1-9e65-05554953e994 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:02.518307 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3e6a9-1988-40ec-a845-376a40c5ebaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:02.519095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3e6a9-1988-40ec-a845-376a40c5ebaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:02.519368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3e6a9-1988-40ec-a845-376a40c5ebaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:02.519633 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3e6a9-1988-40ec-a845-376a40c5ebaf admin admin] PATH_INFO: `/v3/users/3ac5c1df8c174792a45e3637b6262175` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:02.520868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3e6a9-1988-40ec-a845-376a40c5ebaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ac5c1df8c174792a45e3637b6262175)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:02.522906 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3e6a9-1988-40ec-a845-376a40c5ebaf admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:02.605350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-69e3e6a9-1988-40ec-a845-376a40c5ebaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ac5c1df8c174792a45e3637b6262175'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:02.615442 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-69e3e6a9-1988-40ec-a845-376a40c5ebaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ac5c1df8c174792a45e3637b6262175'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:02.627237 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:29:02.635786 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-69e3e6a9-1988-40ec-a845-376a40c5ebaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ac5c1df8c174792a45e3637b6262175'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:02.636777 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-69e3e6a9-1988-40ec-a845-376a40c5ebaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ac5c1df8c174792a45e3637b6262175'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:02.646260 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-69e3e6a9-1988-40ec-a845-376a40c5ebaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ac5c1df8c174792a45e3637b6262175'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:02.654235 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1106/2221] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:29:02 2022] DELETE /identity/v3/users/3ac5c1df8c174792a45e3637b6262175 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:29:02.658896 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdd61b-9b5a-46cf-abb2-6c5910e5fd21 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:02.659516 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdd61b-9b5a-46cf-abb2-6c5910e5fd21 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:02.659516 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdd61b-9b5a-46cf-abb2-6c5910e5fd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:02.679845 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-b7bdd61b-9b5a-46cf-abb2-6c5910e5fd21 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:29:02.681094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b7bdd61b-9b5a-46cf-abb2-6c5910e5fd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:02.750176 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1116/2222] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:29:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:29:02.770278 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3e6a9-1988-40ec-a845-376a40c5ebaf admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:02.771945 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-69e3e6a9-1988-40ec-a845-376a40c5ebaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:02.855582 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3e6a9-1988-40ec-a845-376a40c5ebaf admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:02.859494 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1549ae3-f8ef-42fa-8ce5-9a05fb7929d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:02.860142 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1549ae3-f8ef-42fa-8ce5-9a05fb7929d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:02.860417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1549ae3-f8ef-42fa-8ce5-9a05fb7929d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:02.860692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1549ae3-f8ef-42fa-8ce5-9a05fb7929d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:02.871976 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1549ae3-f8ef-42fa-8ce5-9a05fb7929d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:02.873507 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1549ae3-f8ef-42fa-8ce5-9a05fb7929d3 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:02.887411 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1107/2223] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:29:03.186785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdd61b-9b5a-46cf-abb2-6c5910e5fd21 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:03.188546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b7bdd61b-9b5a-46cf-abb2-6c5910e5fd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:03.202384 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdd61b-9b5a-46cf-abb2-6c5910e5fd21 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:03.207931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe28b7-4e29-4bb5-b7b9-f9669cdb67c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:03.208719 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe28b7-4e29-4bb5-b7b9-f9669cdb67c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:03.209218 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe28b7-4e29-4bb5-b7b9-f9669cdb67c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:03.209689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe28b7-4e29-4bb5-b7b9-f9669cdb67c8 admin admin] PATH_INFO: `/v3/projects/a9f8bd29a3874d7ca3f48ecaead0a62e` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:03.210959 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe28b7-4e29-4bb5-b7b9-f9669cdb67c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9f8bd29a3874d7ca3f48ecaead0a62e)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:03.213066 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe28b7-4e29-4bb5-b7b9-f9669cdb67c8 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:03.252891 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:29:03.266799 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1abe28b7-4e29-4bb5-b7b9-f9669cdb67c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9f8bd29a3874d7ca3f48ecaead0a62e', 'request_id': 'req-1abe28b7-4e29-4bb5-b7b9-f9669cdb67c8'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:03.273424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1abe28b7-4e29-4bb5-b7b9-f9669cdb67c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9f8bd29a3874d7ca3f48ecaead0a62e', 'request_id': 'req-1abe28b7-4e29-4bb5-b7b9-f9669cdb67c8'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:03.282386 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1abe28b7-4e29-4bb5-b7b9-f9669cdb67c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9f8bd29a3874d7ca3f48ecaead0a62e', 'request_id': 'req-1abe28b7-4e29-4bb5-b7b9-f9669cdb67c8'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:03.293783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1abe28b7-4e29-4bb5-b7b9-f9669cdb67c8 admin admin] The token cache is being invalidate because project a9f8bd29a3874d7ca3f48ecaead0a62e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:29:03.294338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1abe28b7-4e29-4bb5-b7b9-f9669cdb67c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:03.295164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1abe28b7-4e29-4bb5-b7b9-f9669cdb67c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:03.297127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1117/2224] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:29:03 2022] DELETE /identity/v3/projects/a9f8bd29a3874d7ca3f48ecaead0a62e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:29:03.302641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dcdb4-7b34-4464-850a-1c2d9252aa8c None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:03.302946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dcdb4-7b34-4464-850a-1c2d9252aa8c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:03.303217 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dcdb4-7b34-4464-850a-1c2d9252aa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:03.323507 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-aa5dcdb4-7b34-4464-850a-1c2d9252aa8c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:29:03.324861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-aa5dcdb4-7b34-4464-850a-1c2d9252aa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:03.389418 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1108/2225] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:29:03.397586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9f4a2-7caa-46c8-b079-4ea192ad2031 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:03.397862 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9f4a2-7caa-46c8-b079-4ea192ad2031 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:03.398133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9f4a2-7caa-46c8-b079-4ea192ad2031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:03.420207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-84f9f4a2-7caa-46c8-b079-4ea192ad2031 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:29:03.421842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-84f9f4a2-7caa-46c8-b079-4ea192ad2031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:03.430674 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1118/2226] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:29:03.435208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5dcdb4-7b34-4464-850a-1c2d9252aa8c None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:03.446742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5dcdb4-7b34-4464-850a-1c2d9252aa8c None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:03.451847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da570f46-5e46-459b-b977-d9b597c1a7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:03.452515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da570f46-5e46-459b-b977-d9b597c1a7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:03.452797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da570f46-5e46-459b-b977-d9b597c1a7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:03.453216 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da570f46-5e46-459b-b977-d9b597c1a7f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:03.453890 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da570f46-5e46-459b-b977-d9b597c1a7f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:29:03.454197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da570f46-5e46-459b-b977-d9b597c1a7f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:03.456035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da570f46-5e46-459b-b977-d9b597c1a7f8 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:03.470844 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1109/2227] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:29:03.475258 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9f4a2-7caa-46c8-b079-4ea192ad2031 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:03.485880 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9f4a2-7caa-46c8-b079-4ea192ad2031 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:03.492242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddea323-e899-42f7-bb8b-791acb51364c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:03.492887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddea323-e899-42f7-bb8b-791acb51364c admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:03.493356 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddea323-e899-42f7-bb8b-791acb51364c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:03.493511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddea323-e899-42f7-bb8b-791acb51364c admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:03.494090 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddea323-e899-42f7-bb8b-791acb51364c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:03.496123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddea323-e899-42f7-bb8b-791acb51364c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:03.531321 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1119/2228] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:29:03 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:29:03.535718 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da570f46-5e46-459b-b977-d9b597c1a7f8 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:03.546872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da570f46-5e46-459b-b977-d9b597c1a7f8 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:03.612348 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33438cea-5ae2-456d-a33d-e894fab00e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:03.612974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33438cea-5ae2-456d-a33d-e894fab00e99 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:03.613253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33438cea-5ae2-456d-a33d-e894fab00e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:03.613525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33438cea-5ae2-456d-a33d-e894fab00e99 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:03.614257 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33438cea-5ae2-456d-a33d-e894fab00e99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:03.631144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33438cea-5ae2-456d-a33d-e894fab00e99 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:03.642951 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1110/2229] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:29:03 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:29:03.647181 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddea323-e899-42f7-bb8b-791acb51364c admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:03.658079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddea323-e899-42f7-bb8b-791acb51364c admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:03.662811 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19772108-5ec8-4d75-ac3d-3b12402829ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:03.663496 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19772108-5ec8-4d75-ac3d-3b12402829ce admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:03.663685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19772108-5ec8-4d75-ac3d-3b12402829ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:03.663950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19772108-5ec8-4d75-ac3d-3b12402829ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:03.664630 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19772108-5ec8-4d75-ac3d-3b12402829ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:03.666222 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19772108-5ec8-4d75-ac3d-3b12402829ce admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:03.677281 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1120/2230] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:29:03.681564 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33438cea-5ae2-456d-a33d-e894fab00e99 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:03.699710 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:29:03.707550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33438cea-5ae2-456d-a33d-e894fab00e99 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:03.715077 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41b496-ccf6-4428-9261-b912954781ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:03.715742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41b496-ccf6-4428-9261-b912954781ff admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:03.716502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41b496-ccf6-4428-9261-b912954781ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:03.717508 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41b496-ccf6-4428-9261-b912954781ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:03.718550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41b496-ccf6-4428-9261-b912954781ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:03.721426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41b496-ccf6-4428-9261-b912954781ff admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:03.732673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1111/2231] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:29:03.737897 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19772108-5ec8-4d75-ac3d-3b12402829ce admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:03.750354 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19772108-5ec8-4d75-ac3d-3b12402829ce admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:03.757658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6be4e-976c-45c9-a6b4-bad469afdf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:03.758350 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6be4e-976c-45c9-a6b4-bad469afdf6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:03.758635 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6be4e-976c-45c9-a6b4-bad469afdf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:03.758902 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6be4e-976c-45c9-a6b4-bad469afdf6a admin admin] PATH_INFO: `/v3/projects/494001e959da4570bf23768a0a47e603/users/d170883bd1b040cfb39c485186734fb4/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:03.776744 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6be4e-976c-45c9-a6b4-bad469afdf6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=494001e959da4570bf23768a0a47e603, user_id=d170883bd1b040cfb39c485186734fb4, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:03.779497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6be4e-976c-45c9-a6b4-bad469afdf6a admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:03.796041 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1121/2232] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:29:03 2022] PUT /identity/v3/projects/494001e959da4570bf23768a0a47e603/users/d170883bd1b040cfb39c485186734fb4/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:29:03.805915 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e53bda-928a-47fa-b5ce-f07f74adb029 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:03.806587 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e53bda-928a-47fa-b5ce-f07f74adb029 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:03.806870 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e53bda-928a-47fa-b5ce-f07f74adb029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:03.838329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-89e53bda-928a-47fa-b5ce-f07f74adb029 None None] MFA Rules not processed for user `d170883bd1b040cfb39c485186734fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:29:03.839775 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-89e53bda-928a-47fa-b5ce-f07f74adb029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:03.936077 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1112/2233] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:29:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:29:03.956422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6be4e-976c-45c9-a6b4-bad469afdf6a admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:03.956422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-27b6be4e-976c-45c9-a6b4-bad469afdf6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:04.058336 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6be4e-976c-45c9-a6b4-bad469afdf6a admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:04.063550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa69dd7-38fc-4121-9162-4445a8d285d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:04.064860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa69dd7-38fc-4121-9162-4445a8d285d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:04.064860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa69dd7-38fc-4121-9162-4445a8d285d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:04.065102 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa69dd7-38fc-4121-9162-4445a8d285d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:04.075236 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa69dd7-38fc-4121-9162-4445a8d285d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:04.076727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa69dd7-38fc-4121-9162-4445a8d285d2 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:04.096178 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1122/2234] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:29:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:29:05.587359 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c6c93-333b-48e1-95b8-35dcb1d4e57d None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:05.587956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c6c93-333b-48e1-95b8-35dcb1d4e57d None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:05.587956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c6c93-333b-48e1-95b8-35dcb1d4e57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:05.610658 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-3b5c6c93-333b-48e1-95b8-35dcb1d4e57d None None] MFA Rules not processed for user `d170883bd1b040cfb39c485186734fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:29:05.612028 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3b5c6c93-333b-48e1-95b8-35dcb1d4e57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:05.620512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1113/2235] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:29:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:29:10.197117 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa69dd7-38fc-4121-9162-4445a8d285d2 service cinder] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:10.198896 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6aa69dd7-38fc-4121-9162-4445a8d285d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:10.269611 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa69dd7-38fc-4121-9162-4445a8d285d2 service cinder] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:10.275636 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c76132-5831-4b44-9851-33e1ccc59479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:10.276711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c76132-5831-4b44-9851-33e1ccc59479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:10.277164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c76132-5831-4b44-9851-33e1ccc59479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:10.277613 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c76132-5831-4b44-9851-33e1ccc59479 admin admin] PATH_INFO: `/v3/users/165e5f6667ae42af9e87fde5d17ccc14` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:10.279348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c76132-5831-4b44-9851-33e1ccc59479 admin admin] RBAC: Authorizing `identity:delete_user(user_id=165e5f6667ae42af9e87fde5d17ccc14)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:10.282505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c76132-5831-4b44-9851-33e1ccc59479 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:10.285079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:29:10.342421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-95c76132-5831-4b44-9851-33e1ccc59479 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '165e5f6667ae42af9e87fde5d17ccc14'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:10.356218 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-95c76132-5831-4b44-9851-33e1ccc59479 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '165e5f6667ae42af9e87fde5d17ccc14'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:10.363089 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-95c76132-5831-4b44-9851-33e1ccc59479 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '165e5f6667ae42af9e87fde5d17ccc14'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:10.363810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-95c76132-5831-4b44-9851-33e1ccc59479 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '165e5f6667ae42af9e87fde5d17ccc14'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:10.364446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-95c76132-5831-4b44-9851-33e1ccc59479 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '165e5f6667ae42af9e87fde5d17ccc14'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:10.377950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1123/2236] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:29:10 2022] DELETE /identity/v3/users/165e5f6667ae42af9e87fde5d17ccc14 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:29:10.382462 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239c74f-2f13-4dcb-a7a4-823e17e4c770 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:10.383003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239c74f-2f13-4dcb-a7a4-823e17e4c770 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:10.383003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239c74f-2f13-4dcb-a7a4-823e17e4c770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:10.404499 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-f239c74f-2f13-4dcb-a7a4-823e17e4c770 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:29:10.405738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f239c74f-2f13-4dcb-a7a4-823e17e4c770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:10.472426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1114/2237] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:29:10.482398 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c76132-5831-4b44-9851-33e1ccc59479 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:10.483967 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-95c76132-5831-4b44-9851-33e1ccc59479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:10.542826 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c76132-5831-4b44-9851-33e1ccc59479 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:10.548117 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b667af88-c280-44c1-8a03-82001245fc9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:10.548718 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667af88-c280-44c1-8a03-82001245fc9a service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:10.548999 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667af88-c280-44c1-8a03-82001245fc9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:10.549273 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667af88-c280-44c1-8a03-82001245fc9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:10.559347 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b667af88-c280-44c1-8a03-82001245fc9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:10.560840 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b667af88-c280-44c1-8a03-82001245fc9a service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:10.571015 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1124/2238] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:29:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:29:10.857791 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239c74f-2f13-4dcb-a7a4-823e17e4c770 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:10.859285 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f239c74f-2f13-4dcb-a7a4-823e17e4c770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:10.871773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239c74f-2f13-4dcb-a7a4-823e17e4c770 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:10.877883 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1dd3f7-e1bd-41a8-ae0c-9124b1249a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:10.878527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dd3f7-e1bd-41a8-ae0c-9124b1249a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:10.878804 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dd3f7-e1bd-41a8-ae0c-9124b1249a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:10.879072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1dd3f7-e1bd-41a8-ae0c-9124b1249a85 admin admin] PATH_INFO: `/v3/projects/cfb89d8a051a4da7850fa596d63c952b` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:10.880434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1dd3f7-e1bd-41a8-ae0c-9124b1249a85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfb89d8a051a4da7850fa596d63c952b)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:10.882377 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1dd3f7-e1bd-41a8-ae0c-9124b1249a85 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:10.933859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-da1dd3f7-e1bd-41a8-ae0c-9124b1249a85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfb89d8a051a4da7850fa596d63c952b', 'request_id': 'req-da1dd3f7-e1bd-41a8-ae0c-9124b1249a85'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:10.939522 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-da1dd3f7-e1bd-41a8-ae0c-9124b1249a85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfb89d8a051a4da7850fa596d63c952b', 'request_id': 'req-da1dd3f7-e1bd-41a8-ae0c-9124b1249a85'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:10.946068 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-da1dd3f7-e1bd-41a8-ae0c-9124b1249a85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfb89d8a051a4da7850fa596d63c952b', 'request_id': 'req-da1dd3f7-e1bd-41a8-ae0c-9124b1249a85'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:10.967517 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:29:10.972392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-da1dd3f7-e1bd-41a8-ae0c-9124b1249a85 admin admin] The token cache is being invalidate because project cfb89d8a051a4da7850fa596d63c952b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:29:10.972730 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-da1dd3f7-e1bd-41a8-ae0c-9124b1249a85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:10.974016 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-da1dd3f7-e1bd-41a8-ae0c-9124b1249a85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:10.976335 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1115/2239] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:29:10 2022] DELETE /identity/v3/projects/cfb89d8a051a4da7850fa596d63c952b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:29:20.566286 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=52,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:29:21.683904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b667af88-c280-44c1-8a03-82001245fc9a service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:21.685553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b667af88-c280-44c1-8a03-82001245fc9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:21.759481 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b667af88-c280-44c1-8a03-82001245fc9a service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:21.823329 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c8452-1933-4aae-b36c-312a54fce3a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '143c30b4201645279564e72ab3288adc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:21.823972 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c8452-1933-4aae-b36c-312a54fce3a0 service placement] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:21.824285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c8452-1933-4aae-b36c-312a54fce3a0 service placement] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:21.824557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9c8452-1933-4aae-b36c-312a54fce3a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:21.826398 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-fb9c8452-1933-4aae-b36c-312a54fce3a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:21.838220 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c8452-1933-4aae-b36c-312a54fce3a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:21.839821 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9c8452-1933-4aae-b36c-312a54fce3a0 service placement] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:21.850153 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1125/2240] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:29:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:29:23.189559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023d8b0-8567-41a3-b213-7891d04f8da1 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:23.190088 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023d8b0-8567-41a3-b213-7891d04f8da1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:23.190088 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023d8b0-8567-41a3-b213-7891d04f8da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:23.209897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-8023d8b0-8567-41a3-b213-7891d04f8da1 None None] MFA Rules not processed for user `61ff29258a214817b8fc15b51fe8b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:29:23.211114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8023d8b0-8567-41a3-b213-7891d04f8da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:23.268469 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1116/2241] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Aug 7 19:29:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:29:23.310654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c8452-1933-4aae-b36c-312a54fce3a0 service placement] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:23.312244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-fb9c8452-1933-4aae-b36c-312a54fce3a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:23.383537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9c8452-1933-4aae-b36c-312a54fce3a0 service placement] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:23.389444 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc8cae-178d-4ae4-bdc6-557052199b7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db51971158a946c98a8ed716ecc5d72b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:23.390059 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc8cae-178d-4ae4-bdc6-557052199b7e service swift] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:23.390340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc8cae-178d-4ae4-bdc6-557052199b7e service swift] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:23.390621 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc8cae-178d-4ae4-bdc6-557052199b7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:23.399540 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc8cae-178d-4ae4-bdc6-557052199b7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:23.401172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc8cae-178d-4ae4-bdc6-557052199b7e service swift] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:23.410495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1126/2242] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Aug 7 19:29:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 19:29:23.979557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b019554-c515-4293-9382-4032e3cafb97 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:23.979918 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b019554-c515-4293-9382-4032e3cafb97 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:23.980221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b019554-c515-4293-9382-4032e3cafb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:24.000332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-5b019554-c515-4293-9382-4032e3cafb97 None None] MFA Rules not processed for user `61ff29258a214817b8fc15b51fe8b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:29:24.001546 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-5b019554-c515-4293-9382-4032e3cafb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:24.009107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1117/2243] 10.222.0.17 () {58 vars in 967 bytes} [Sun Aug 7 19:29:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:29:24.023007 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc8cae-178d-4ae4-bdc6-557052199b7e service swift] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:24.033920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc8cae-178d-4ae4-bdc6-557052199b7e service swift] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:24.039420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5ff1b-109b-486b-9551-5c868ac5ec65 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db51971158a946c98a8ed716ecc5d72b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:24.040035 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5ff1b-109b-486b-9551-5c868ac5ec65 service swift] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:24.040332 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5ff1b-109b-486b-9551-5c868ac5ec65 service swift] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:24.040607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5ff1b-109b-486b-9551-5c868ac5ec65 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:24.049380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5ff1b-109b-486b-9551-5c868ac5ec65 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:24.050963 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5ff1b-109b-486b-9551-5c868ac5ec65 service swift] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:24.059249 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1127/2244] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Aug 7 19:29:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 07 19:29:24.310151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12345c-74a4-4112-97d0-a23b58e78770 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:24.310633 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12345c-74a4-4112-97d0-a23b58e78770 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:24.310725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12345c-74a4-4112-97d0-a23b58e78770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:24.330963 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-ff12345c-74a4-4112-97d0-a23b58e78770 None None] MFA Rules not processed for user `61ff29258a214817b8fc15b51fe8b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:29:24.332220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-ff12345c-74a4-4112-97d0-a23b58e78770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:24.339920 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1118/2245] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Aug 7 19:29:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:29:24.353149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5ff1b-109b-486b-9551-5c868ac5ec65 service swift] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:24.364308 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5ff1b-109b-486b-9551-5c868ac5ec65 service swift] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:24.370520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033bd826-677f-434c-9bfc-401eaf6fbb7b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db51971158a946c98a8ed716ecc5d72b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:24.371188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bd826-677f-434c-9bfc-401eaf6fbb7b service swift] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:24.371601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bd826-677f-434c-9bfc-401eaf6fbb7b service swift] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:24.371881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bd826-677f-434c-9bfc-401eaf6fbb7b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:24.378639 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:29:24.380978 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033bd826-677f-434c-9bfc-401eaf6fbb7b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:24.382942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033bd826-677f-434c-9bfc-401eaf6fbb7b service swift] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:24.391666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1128/2246] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Aug 7 19:29:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 19:29:24.752919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b10ad3-183a-4876-98b2-b6d5e213d63a None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:24.753192 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b10ad3-183a-4876-98b2-b6d5e213d63a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:24.753463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b10ad3-183a-4876-98b2-b6d5e213d63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:24.773846 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-02b10ad3-183a-4876-98b2-b6d5e213d63a None None] MFA Rules not processed for user `61ff29258a214817b8fc15b51fe8b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:29:24.775056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-02b10ad3-183a-4876-98b2-b6d5e213d63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:24.782536 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1119/2247] 10.222.0.17 () {58 vars in 967 bytes} [Sun Aug 7 19:29:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:29:24.798039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033bd826-677f-434c-9bfc-401eaf6fbb7b service swift] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:24.809281 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033bd826-677f-434c-9bfc-401eaf6fbb7b service swift] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:24.815487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f922a9b-8875-45f8-bdc4-bbf1ca58a3c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db51971158a946c98a8ed716ecc5d72b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:24.816125 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f922a9b-8875-45f8-bdc4-bbf1ca58a3c1 service swift] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:24.816411 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f922a9b-8875-45f8-bdc4-bbf1ca58a3c1 service swift] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:24.816682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f922a9b-8875-45f8-bdc4-bbf1ca58a3c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:24.825750 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f922a9b-8875-45f8-bdc4-bbf1ca58a3c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:24.827339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f922a9b-8875-45f8-bdc4-bbf1ca58a3c1 service swift] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:24.835819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1129/2248] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Aug 7 19:29:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 19:29:27.028910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b10ad3-183a-4876-98b2-b6d5e213d63a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:27.030605 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-02b10ad3-183a-4876-98b2-b6d5e213d63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:27.101209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b10ad3-183a-4876-98b2-b6d5e213d63a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:27.106645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b1a57-2088-4655-ae31-c93220b857c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:27.107231 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b1a57-2088-4655-ae31-c93220b857c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:27.107510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b1a57-2088-4655-ae31-c93220b857c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:27.107777 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581b1a57-2088-4655-ae31-c93220b857c1 admin admin] PATH_INFO: `/v3/users/d170883bd1b040cfb39c485186734fb4` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:27.109079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b1a57-2088-4655-ae31-c93220b857c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d170883bd1b040cfb39c485186734fb4)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:27.111082 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581b1a57-2088-4655-ae31-c93220b857c1 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:27.159388 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-581b1a57-2088-4655-ae31-c93220b857c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd170883bd1b040cfb39c485186734fb4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:27.166462 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-581b1a57-2088-4655-ae31-c93220b857c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd170883bd1b040cfb39c485186734fb4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:27.178812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-581b1a57-2088-4655-ae31-c93220b857c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd170883bd1b040cfb39c485186734fb4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:27.179526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-581b1a57-2088-4655-ae31-c93220b857c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd170883bd1b040cfb39c485186734fb4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:27.187617 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-581b1a57-2088-4655-ae31-c93220b857c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd170883bd1b040cfb39c485186734fb4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:27.195118 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1120/2249] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:29:27 2022] DELETE /identity/v3/users/d170883bd1b040cfb39c485186734fb4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:29:27.200134 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584b13b-b066-4bde-9470-00c540a3c8ec None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:27.200662 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584b13b-b066-4bde-9470-00c540a3c8ec None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:27.200662 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584b13b-b066-4bde-9470-00c540a3c8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:27.222319 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-e584b13b-b066-4bde-9470-00c540a3c8ec None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:29:27.223601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e584b13b-b066-4bde-9470-00c540a3c8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:27.295709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1130/2250] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:29:27.306879 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b1a57-2088-4655-ae31-c93220b857c1 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:27.308460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-581b1a57-2088-4655-ae31-c93220b857c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:27.316049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:29:27.376860 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581b1a57-2088-4655-ae31-c93220b857c1 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:27.381035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1b9a00-9e68-4218-b98a-5ee72bcd2d69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:27.381711 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b9a00-9e68-4218-b98a-5ee72bcd2d69 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:27.381967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b9a00-9e68-4218-b98a-5ee72bcd2d69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:27.382230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1b9a00-9e68-4218-b98a-5ee72bcd2d69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:27.392955 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b9a00-9e68-4218-b98a-5ee72bcd2d69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:27.394450 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1b9a00-9e68-4218-b98a-5ee72bcd2d69 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:27.405807 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1121/2251] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:29:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:29:27.584944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584b13b-b066-4bde-9470-00c540a3c8ec None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:27.586472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e584b13b-b066-4bde-9470-00c540a3c8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:27.599858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584b13b-b066-4bde-9470-00c540a3c8ec None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:27.605739 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480fdb60-0795-4a69-9858-d78375c9958f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:27.606483 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480fdb60-0795-4a69-9858-d78375c9958f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:27.606849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480fdb60-0795-4a69-9858-d78375c9958f admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:27.607189 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480fdb60-0795-4a69-9858-d78375c9958f admin admin] PATH_INFO: `/v3/projects/494001e959da4570bf23768a0a47e603` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:27.608601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480fdb60-0795-4a69-9858-d78375c9958f admin admin] RBAC: Authorizing `identity:delete_project(project_id=494001e959da4570bf23768a0a47e603)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:27.611428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480fdb60-0795-4a69-9858-d78375c9958f admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:27.665343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-480fdb60-0795-4a69-9858-d78375c9958f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '494001e959da4570bf23768a0a47e603', 'request_id': 'req-480fdb60-0795-4a69-9858-d78375c9958f'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:27.672574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-480fdb60-0795-4a69-9858-d78375c9958f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '494001e959da4570bf23768a0a47e603', 'request_id': 'req-480fdb60-0795-4a69-9858-d78375c9958f'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:27.681548 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-480fdb60-0795-4a69-9858-d78375c9958f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '494001e959da4570bf23768a0a47e603', 'request_id': 'req-480fdb60-0795-4a69-9858-d78375c9958f'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:27.693030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-480fdb60-0795-4a69-9858-d78375c9958f admin admin] The token cache is being invalidate because project 494001e959da4570bf23768a0a47e603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:29:27.693392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-480fdb60-0795-4a69-9858-d78375c9958f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:27.694090 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-480fdb60-0795-4a69-9858-d78375c9958f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:29:27.695668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1131/2252] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:29:27 2022] DELETE /identity/v3/projects/494001e959da4570bf23768a0a47e603 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:29:37.400618 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=24 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:29:37.689570 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=62,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:29:53.139531 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac9cd5-4b00-49a8-a378-01d64d908277 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:53.140148 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac9cd5-4b00-49a8-a378-01d64d908277 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:53.140148 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac9cd5-4b00-49a8-a378-01d64d908277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:53.164089 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-57ac9cd5-4b00-49a8-a378-01d64d908277 None None] MFA Rules not processed for user `61ff29258a214817b8fc15b51fe8b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:29:53.165317 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-57ac9cd5-4b00-49a8-a378-01d64d908277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:53.235608 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1122/2253] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Aug 7 19:29:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:29:53.249095 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480fdb60-0795-4a69-9858-d78375c9958f admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:29:53.250517 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-480fdb60-0795-4a69-9858-d78375c9958f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:29:53.325009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480fdb60-0795-4a69-9858-d78375c9958f admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:29:53.331516 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40eedd-6bb4-4999-9582-09fdf412f845 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db51971158a946c98a8ed716ecc5d72b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:29:53.332160 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40eedd-6bb4-4999-9582-09fdf412f845 service swift] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:29:53.332437 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40eedd-6bb4-4999-9582-09fdf412f845 service swift] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:29:53.332708 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40eedd-6bb4-4999-9582-09fdf412f845 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:29:53.342310 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40eedd-6bb4-4999-9582-09fdf412f845 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:29:53.343953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40eedd-6bb4-4999-9582-09fdf412f845 service swift] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:29:53.353943 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1132/2254] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Aug 7 19:29:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 19:30:02.246716 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b58cce-9a2b-4210-8871-4fd103ced21c None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:30:02.246716 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b58cce-9a2b-4210-8871-4fd103ced21c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:30:02.246716 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b58cce-9a2b-4210-8871-4fd103ced21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:30:02.283137 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-e0b58cce-9a2b-4210-8871-4fd103ced21c None None] MFA Rules not processed for user `be3e2afc428a45e2b993c39e3d53cd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:30:02.284501 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e0b58cce-9a2b-4210-8871-4fd103ced21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:30:02.390998 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1123/2255] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:30:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:30:03.349720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=32 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:30:08.758603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebf2a6-fb2f-461d-bc6c-717b2760eff7 None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:30:08.759678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebf2a6-fb2f-461d-bc6c-717b2760eff7 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:30:08.760201 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebf2a6-fb2f-461d-bc6c-717b2760eff7 None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:30:08.762009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1133/2256] 10.222.0.17 () {54 vars in 921 bytes} [Sun Aug 7 19:30:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:30:08.768471 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c20bf-7e61-4422-b220-e6dee648a083 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:30:08.769401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c20bf-7e61-4422-b220-e6dee648a083 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:30:08.769775 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c20bf-7e61-4422-b220-e6dee648a083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:30:08.794853 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-204c20bf-7e61-4422-b220-e6dee648a083 None None] MFA Rules not processed for user `a4946aec299f45bd8402cff3159d04d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:30:08.796868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-204c20bf-7e61-4422-b220-e6dee648a083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:30:08.924453 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1124/2257] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Aug 7 19:30:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:30:08.936832 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebf2a6-fb2f-461d-bc6c-717b2760eff7 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:30:08.939251 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4bebf2a6-fb2f-461d-bc6c-717b2760eff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:30:09.027899 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebf2a6-fb2f-461d-bc6c-717b2760eff7 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:30:09.031887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f2d18-9cf0-465a-a456-a7e1a12f52c0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9daa1a3c7654dcfa5ccb03f467ce7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:30:09.032564 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f2d18-9cf0-465a-a456-a7e1a12f52c0 service glance] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:30:09.032847 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f2d18-9cf0-465a-a456-a7e1a12f52c0 service glance] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:30:09.033118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f2d18-9cf0-465a-a456-a7e1a12f52c0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:30:09.045290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f2d18-9cf0-465a-a456-a7e1a12f52c0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:30:09.047153 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30f2d18-9cf0-465a-a456-a7e1a12f52c0 service glance] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:30:09.061482 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1134/2258] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:30:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:30:09.149490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d5344-42d5-434a-aab4-566cb65015ba None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:30:09.150044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d5344-42d5-434a-aab4-566cb65015ba None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:30:09.150790 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d5344-42d5-434a-aab4-566cb65015ba None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:30:09.153364 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1125/2259] 10.222.0.17 () {54 vars in 921 bytes} [Sun Aug 7 19:30:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:30:09.163000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912409b-ff11-4962-ad58-6f659f061b66 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:30:09.163330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912409b-ff11-4962-ad58-6f659f061b66 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:30:09.163524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912409b-ff11-4962-ad58-6f659f061b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:30:09.186669 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-b912409b-ff11-4962-ad58-6f659f061b66 None None] MFA Rules not processed for user `08229e9400b14e8a9cc8e8c32dc48285`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:30:09.187794 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b912409b-ff11-4962-ad58-6f659f061b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:30:09.253478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1135/2260] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Aug 7 19:30:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:30:09.285946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d5344-42d5-434a-aab4-566cb65015ba None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:30:09.287734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e05d5344-42d5-434a-aab4-566cb65015ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:30:09.302704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d5344-42d5-434a-aab4-566cb65015ba None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:30:09.306883 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff9bc9-5820-4fed-bfaf-1c5c813fc7f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:30:09.307819 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff9bc9-5820-4fed-bfaf-1c5c813fc7f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:30:09.308393 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff9bc9-5820-4fed-bfaf-1c5c813fc7f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:30:09.308837 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff9bc9-5820-4fed-bfaf-1c5c813fc7f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:30:09.318340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff9bc9-5820-4fed-bfaf-1c5c813fc7f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:30:09.319033 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff9bc9-5820-4fed-bfaf-1c5c813fc7f4 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:30:09.327642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1126/2261] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:30:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:30:09.465436 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c67b45-1f0b-4f06-83c9-f320dd981efc None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:30:09.465711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c67b45-1f0b-4f06-83c9-f320dd981efc None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:30:09.465986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c67b45-1f0b-4f06-83c9-f320dd981efc None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:30:09.467348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1136/2262] 10.222.0.17 () {54 vars in 921 bytes} [Sun Aug 7 19:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 19:30:09.473391 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed135543-3198-4e62-ac25-ac7563c0445b None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:30:09.473791 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed135543-3198-4e62-ac25-ac7563c0445b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:30:09.474182 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed135543-3198-4e62-ac25-ac7563c0445b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:30:09.495683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-ed135543-3198-4e62-ac25-ac7563c0445b None None] MFA Rules not processed for user `a4946aec299f45bd8402cff3159d04d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:30:09.497021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-ed135543-3198-4e62-ac25-ac7563c0445b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:30:09.516972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1127/2263] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Aug 7 19:30:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:30:09.531338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c67b45-1f0b-4f06-83c9-f320dd981efc None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:30:09.546594 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c67b45-1f0b-4f06-83c9-f320dd981efc None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:30:09.551610 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697d135-7acd-4b54-b448-1f7da9f8d58d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:30:09.552273 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697d135-7acd-4b54-b448-1f7da9f8d58d service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:30:09.552616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697d135-7acd-4b54-b448-1f7da9f8d58d service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:30:09.552836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697d135-7acd-4b54-b448-1f7da9f8d58d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:30:09.563618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697d135-7acd-4b54-b448-1f7da9f8d58d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:30:09.565093 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697d135-7acd-4b54-b448-1f7da9f8d58d service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:30:09.577002 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1137/2264] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:30:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:30:13.661043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fda876-cb4e-4157-b947-77cb8e28e682 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:30:13.661535 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fda876-cb4e-4157-b947-77cb8e28e682 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:30:13.661733 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fda876-cb4e-4157-b947-77cb8e28e682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:30:13.685849 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-96fda876-cb4e-4157-b947-77cb8e28e682 None None] MFA Rules not processed for user `61ff29258a214817b8fc15b51fe8b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:30:13.687068 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-96fda876-cb4e-4157-b947-77cb8e28e682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:30:13.694327 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1128/2265] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Aug 7 19:30:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:30:13.707294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697d135-7acd-4b54-b448-1f7da9f8d58d service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:30:13.718577 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697d135-7acd-4b54-b448-1f7da9f8d58d service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:30:13.724097 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d96ba-3769-4afa-8eed-ef33ef813895 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db51971158a946c98a8ed716ecc5d72b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:30:13.724702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d96ba-3769-4afa-8eed-ef33ef813895 service swift] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:30:13.724978 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d96ba-3769-4afa-8eed-ef33ef813895 service swift] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:30:13.725244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d96ba-3769-4afa-8eed-ef33ef813895 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:30:13.733860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d96ba-3769-4afa-8eed-ef33ef813895 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:30:13.735453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d96ba-3769-4afa-8eed-ef33ef813895 service swift] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:30:13.743983 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1138/2266] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Aug 7 19:30:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 19:30:23.680427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=93 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:30:23.740500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=61 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:30:30.131971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb654f-904b-4bc6-942b-262e5dfce58a None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:30:30.132714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb654f-904b-4bc6-942b-262e5dfce58a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:30:30.132714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb654f-904b-4bc6-942b-262e5dfce58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:30:30.154575 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-1afb654f-904b-4bc6-942b-262e5dfce58a None None] MFA Rules not processed for user `f643fd4ee6bc4ccfae45f3b8523ae46a`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:30:30.156110 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-1afb654f-904b-4bc6-942b-262e5dfce58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:30:30.228917 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1129/2267] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:30:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:30:31.813664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30841425-af01-4eef-9245-65b59274775b None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:30:31.814194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30841425-af01-4eef-9245-65b59274775b None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:30:31.814194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30841425-af01-4eef-9245-65b59274775b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:30:31.837044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-30841425-af01-4eef-9245-65b59274775b None None] MFA Rules not processed for user `61ff29258a214817b8fc15b51fe8b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:30:31.838302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-30841425-af01-4eef-9245-65b59274775b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:30:31.848506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1139/2268] 10.222.0.17 () {58 vars in 967 bytes} [Sun Aug 7 19:30:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:30:31.877361 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb654f-904b-4bc6-942b-262e5dfce58a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:30:31.891782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb654f-904b-4bc6-942b-262e5dfce58a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:30:31.900132 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6dcf5-e56f-46e4-8d85-780c084b547f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db51971158a946c98a8ed716ecc5d72b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:30:31.901356 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6dcf5-e56f-46e4-8d85-780c084b547f service swift] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:30:31.902013 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6dcf5-e56f-46e4-8d85-780c084b547f service swift] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:30:31.902522 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6dcf5-e56f-46e4-8d85-780c084b547f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:30:31.916015 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6dcf5-e56f-46e4-8d85-780c084b547f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:30:31.917914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6dcf5-e56f-46e4-8d85-780c084b547f service swift] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:30:31.927450 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1130/2269] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Aug 7 19:30:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 19:30:41.831901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:30:41.923774 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=31 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:30:53.090760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ef771-147b-456e-9729-fd88a050c929 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:30:53.091328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ef771-147b-456e-9729-fd88a050c929 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:30:53.091328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ef771-147b-456e-9729-fd88a050c929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:30:53.110767 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-2e1ef771-147b-456e-9729-fd88a050c929 None None] MFA Rules not processed for user `6732487041c54918b1164b1cb72bb07e`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:30:53.112019 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-2e1ef771-147b-456e-9729-fd88a050c929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:30:53.168507 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1140/2270] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:30:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:30:59.811881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6dcf5-e56f-46e4-8d85-780c084b547f service swift] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:30:59.825482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6dcf5-e56f-46e4-8d85-780c084b547f service swift] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:30:59.830846 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf7643-e77b-4a90-bb7c-da3eba9121c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:30:59.831515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf7643-e77b-4a90-bb7c-da3eba9121c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:30:59.831795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf7643-e77b-4a90-bb7c-da3eba9121c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:30:59.832110 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf7643-e77b-4a90-bb7c-da3eba9121c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:30:59.833976 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-67cf7643-e77b-4a90-bb7c-da3eba9121c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:30:59.847716 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf7643-e77b-4a90-bb7c-da3eba9121c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:30:59.849194 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf7643-e77b-4a90-bb7c-da3eba9121c7 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:30:59.860714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1131/2271] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:30:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:31:01.049761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ef771-147b-456e-9729-fd88a050c929 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:01.062499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ef771-147b-456e-9729-fd88a050c929 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:01.067160 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5cf8f-55f8-4b54-9921-ed79b48caa0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:01.067854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5cf8f-55f8-4b54-9921-ed79b48caa0f service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:01.068310 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5cf8f-55f8-4b54-9921-ed79b48caa0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:01.068596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c5cf8f-55f8-4b54-9921-ed79b48caa0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:01.070603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f7c5cf8f-55f8-4b54-9921-ed79b48caa0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:01.087286 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5cf8f-55f8-4b54-9921-ed79b48caa0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:01.088713 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c5cf8f-55f8-4b54-9921-ed79b48caa0f service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:01.099721 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1141/2272] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:31:09.855477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=6 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:31:11.094834 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=31 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:31:12.471772 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf7643-e77b-4a90-bb7c-da3eba9121c7 service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:12.474237 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-67cf7643-e77b-4a90-bb7c-da3eba9121c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:12.492705 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf7643-e77b-4a90-bb7c-da3eba9121c7 service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:12.500412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8fe9e-7068-438d-85d6-85ad6f66b8cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:12.501026 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8fe9e-7068-438d-85d6-85ad6f66b8cd service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:12.501306 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8fe9e-7068-438d-85d6-85ad6f66b8cd service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:12.501580 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8fe9e-7068-438d-85d6-85ad6f66b8cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:12.503560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-54f8fe9e-7068-438d-85d6-85ad6f66b8cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:12.514473 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f8fe9e-7068-438d-85d6-85ad6f66b8cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:12.516185 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f8fe9e-7068-438d-85d6-85ad6f66b8cd service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:12.525370 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1132/2273] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:31:16.107131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5cf8f-55f8-4b54-9921-ed79b48caa0f service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:16.108821 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f7c5cf8f-55f8-4b54-9921-ed79b48caa0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:16.168301 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c5cf8f-55f8-4b54-9921-ed79b48caa0f service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:16.172698 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb71aac-7403-4313-b2fd-54d953683e02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:16.173318 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb71aac-7403-4313-b2fd-54d953683e02 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:16.173603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb71aac-7403-4313-b2fd-54d953683e02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:16.173879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb71aac-7403-4313-b2fd-54d953683e02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:16.175742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8fb71aac-7403-4313-b2fd-54d953683e02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:16.187424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb71aac-7403-4313-b2fd-54d953683e02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:16.189690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb71aac-7403-4313-b2fd-54d953683e02 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:16.203780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1142/2274] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:16 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:31:22.311090 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8c6d7-5725-4465-89c4-416e9e925c35 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:22.311657 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8c6d7-5725-4465-89c4-416e9e925c35 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:22.311657 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8c6d7-5725-4465-89c4-416e9e925c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:22.331389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-89c8c6d7-5725-4465-89c4-416e9e925c35 None None] MFA Rules not processed for user `bb092d77de334d3ab631a42979460c59`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:31:22.332724 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-89c8c6d7-5725-4465-89c4-416e9e925c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:22.389751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1133/2275] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:31:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:31:22.708218 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb71aac-7403-4313-b2fd-54d953683e02 service cinder] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:22.718128 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb71aac-7403-4313-b2fd-54d953683e02 service cinder] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:22.722394 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80993c-606a-43ef-b822-275333a2ab49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:22.723099 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80993c-606a-43ef-b822-275333a2ab49 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:22.723571 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80993c-606a-43ef-b822-275333a2ab49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:22.724033 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80993c-606a-43ef-b822-275333a2ab49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:22.725826 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-bb80993c-606a-43ef-b822-275333a2ab49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:22.735953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb80993c-606a-43ef-b822-275333a2ab49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:22.737424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb80993c-606a-43ef-b822-275333a2ab49 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:22.746183 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1143/2276] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:31:23.128669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8c6d7-5725-4465-89c4-416e9e925c35 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:23.130256 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-89c8c6d7-5725-4465-89c4-416e9e925c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:23.205304 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8c6d7-5725-4465-89c4-416e9e925c35 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:23.210420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642f0b8-0d2e-40da-864c-f3a600fb9089 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '143c30b4201645279564e72ab3288adc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:23.211043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642f0b8-0d2e-40da-864c-f3a600fb9089 service placement] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:23.211314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642f0b8-0d2e-40da-864c-f3a600fb9089 service placement] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:23.211583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642f0b8-0d2e-40da-864c-f3a600fb9089 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:23.213385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-2642f0b8-0d2e-40da-864c-f3a600fb9089 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:23.225371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642f0b8-0d2e-40da-864c-f3a600fb9089 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:23.226964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642f0b8-0d2e-40da-864c-f3a600fb9089 service placement] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:23.237305 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1134/2277] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:31:26.916177 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80993c-606a-43ef-b822-275333a2ab49 service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:26.918138 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-bb80993c-606a-43ef-b822-275333a2ab49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:26.938930 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80993c-606a-43ef-b822-275333a2ab49 service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:26.946879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c074a-7a0a-4164-8acc-8bc5c12baea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:26.948228 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c074a-7a0a-4164-8acc-8bc5c12baea4 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:26.948730 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c074a-7a0a-4164-8acc-8bc5c12baea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:26.949340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c074a-7a0a-4164-8acc-8bc5c12baea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:26.968037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8a8c074a-7a0a-4164-8acc-8bc5c12baea4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:26.988440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c074a-7a0a-4164-8acc-8bc5c12baea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:26.990371 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c074a-7a0a-4164-8acc-8bc5c12baea4 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:27.000500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1144/2278] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:31:32.225779 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642f0b8-0d2e-40da-864c-f3a600fb9089 service placement] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:32.235660 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642f0b8-0d2e-40da-864c-f3a600fb9089 service placement] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:32.239416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aaa518-9320-4175-b045-39dd923f5f43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:32.240170 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aaa518-9320-4175-b045-39dd923f5f43 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:32.240451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aaa518-9320-4175-b045-39dd923f5f43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:32.240718 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aaa518-9320-4175-b045-39dd923f5f43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:32.242561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-70aaa518-9320-4175-b045-39dd923f5f43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:32.253480 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aaa518-9320-4175-b045-39dd923f5f43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:32.254937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aaa518-9320-4175-b045-39dd923f5f43 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:32.264178 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1135/2279] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:31:32.427275 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa816c-3b51-479e-9091-48773dfbca1c None None] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:32.427863 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa816c-3b51-479e-9091-48773dfbca1c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:32.428194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa816c-3b51-479e-9091-48773dfbca1c None None] PATH_INFO: `/` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:32.429935 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1145/2280] 10.222.0.17 () {54 vars in 921 bytes} [Sun Aug 7 19:31:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:31:32.436788 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf100b8-64e0-4ef3-a451-3785a979f1b7 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:32.438394 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf100b8-64e0-4ef3-a451-3785a979f1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:32.438873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf100b8-64e0-4ef3-a451-3785a979f1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:32.459822 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-7cf100b8-64e0-4ef3-a451-3785a979f1b7 None None] MFA Rules not processed for user `08229e9400b14e8a9cc8e8c32dc48285`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:31:32.461355 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-7cf100b8-64e0-4ef3-a451-3785a979f1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:32.468105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1136/2281] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Aug 7 19:31:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:31:32.480364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa816c-3b51-479e-9091-48773dfbca1c None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:32.497458 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa816c-3b51-479e-9091-48773dfbca1c None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:32.501124 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12048633-a8d5-4567-8009-39788800625b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:32.501728 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12048633-a8d5-4567-8009-39788800625b service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:32.502006 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12048633-a8d5-4567-8009-39788800625b service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:32.502277 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12048633-a8d5-4567-8009-39788800625b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:32.509837 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12048633-a8d5-4567-8009-39788800625b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:32.511263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12048633-a8d5-4567-8009-39788800625b service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:32.519861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1146/2282] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:31:32.611698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7ebc5-0e8f-4d77-adc3-a1dc7055416f None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:32.612310 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7ebc5-0e8f-4d77-adc3-a1dc7055416f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:32.612586 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7ebc5-0e8f-4d77-adc3-a1dc7055416f None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:32.614198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1137/2283] 10.222.0.17 () {54 vars in 921 bytes} [Sun Aug 7 19:31:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:31:32.620045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6036e75-6ce3-48b0-adb7-49d59ce02f95 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:32.620405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6036e75-6ce3-48b0-adb7-49d59ce02f95 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:32.620746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6036e75-6ce3-48b0-adb7-49d59ce02f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:32.640545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-f6036e75-6ce3-48b0-adb7-49d59ce02f95 None None] MFA Rules not processed for user `a4946aec299f45bd8402cff3159d04d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:31:32.641940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f6036e75-6ce3-48b0-adb7-49d59ce02f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:32.653120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1147/2284] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Aug 7 19:31:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:31:32.677431 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7ebc5-0e8f-4d77-adc3-a1dc7055416f None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:32.689357 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7ebc5-0e8f-4d77-adc3-a1dc7055416f None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:32.695323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108f843-675f-4b35-b7ca-c5ba2e735a88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:32.695961 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108f843-675f-4b35-b7ca-c5ba2e735a88 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:32.696298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108f843-675f-4b35-b7ca-c5ba2e735a88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:32.696601 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108f843-675f-4b35-b7ca-c5ba2e735a88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:32.707159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108f843-675f-4b35-b7ca-c5ba2e735a88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:32.708643 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108f843-675f-4b35-b7ca-c5ba2e735a88 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:32.719694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1138/2285] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:31:33.005175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05b523-a2af-42c6-b4d9-8eb8fb1fa256 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:33.005466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05b523-a2af-42c6-b4d9-8eb8fb1fa256 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:33.005742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05b523-a2af-42c6-b4d9-8eb8fb1fa256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:33.026595 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-ec05b523-a2af-42c6-b4d9-8eb8fb1fa256 None None] MFA Rules not processed for user `be3e2afc428a45e2b993c39e3d53cd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:31:33.027720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ec05b523-a2af-42c6-b4d9-8eb8fb1fa256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:33.034916 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1148/2286] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:31:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:31:36.990239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108f843-675f-4b35-b7ca-c5ba2e735a88 service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:37.001918 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108f843-675f-4b35-b7ca-c5ba2e735a88 service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:37.016400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35368780-280f-4641-8511-82404b10c944 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:37.017151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35368780-280f-4641-8511-82404b10c944 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:37.017520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35368780-280f-4641-8511-82404b10c944 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:37.017913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35368780-280f-4641-8511-82404b10c944 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:37.019946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-35368780-280f-4641-8511-82404b10c944 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:37.034029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35368780-280f-4641-8511-82404b10c944 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:37.036651 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35368780-280f-4641-8511-82404b10c944 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:37.050488 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1139/2287] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:31:43.021257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=50 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:31:43.529819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05b523-a2af-42c6-b4d9-8eb8fb1fa256 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:43.541117 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05b523-a2af-42c6-b4d9-8eb8fb1fa256 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:43.546044 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abcf5e-8cf6-4160-aa5a-71ba08d9bf84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:43.546715 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abcf5e-8cf6-4160-aa5a-71ba08d9bf84 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:43.546993 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abcf5e-8cf6-4160-aa5a-71ba08d9bf84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:43.547265 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abcf5e-8cf6-4160-aa5a-71ba08d9bf84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:43.549326 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-37abcf5e-8cf6-4160-aa5a-71ba08d9bf84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:43.621348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abcf5e-8cf6-4160-aa5a-71ba08d9bf84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:43.623473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abcf5e-8cf6-4160-aa5a-71ba08d9bf84 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:43.640822 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1149/2288] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:31:44.389348 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35368780-280f-4641-8511-82404b10c944 service nova] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:44.391458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-35368780-280f-4641-8511-82404b10c944 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:44.400224 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=85 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:31:44.411290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35368780-280f-4641-8511-82404b10c944 service nova] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:44.421377 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ee14d-94dc-4b39-995a-a0c21f356267 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:44.423517 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ee14d-94dc-4b39-995a-a0c21f356267 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:44.424275 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ee14d-94dc-4b39-995a-a0c21f356267 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:44.424275 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1ee14d-94dc-4b39-995a-a0c21f356267 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:44.427292 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4d1ee14d-94dc-4b39-995a-a0c21f356267 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:44.443076 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ee14d-94dc-4b39-995a-a0c21f356267 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:44.445152 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1ee14d-94dc-4b39-995a-a0c21f356267 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:44.460611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1140/2289] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:31:44.594357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abcf5e-8cf6-4160-aa5a-71ba08d9bf84 service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:44.619257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abcf5e-8cf6-4160-aa5a-71ba08d9bf84 service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:44.631964 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d46c87-37fa-4ff6-be3b-cde01faeafd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:44.632637 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d46c87-37fa-4ff6-be3b-cde01faeafd2 service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:44.632920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d46c87-37fa-4ff6-be3b-cde01faeafd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:44.633193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d46c87-37fa-4ff6-be3b-cde01faeafd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:44.637823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e0d46c87-37fa-4ff6-be3b-cde01faeafd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:44.668465 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d46c87-37fa-4ff6-be3b-cde01faeafd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:44.670158 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d46c87-37fa-4ff6-be3b-cde01faeafd2 service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:44.695729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1150/2290] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:31:47.876576 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ee14d-94dc-4b39-995a-a0c21f356267 service nova] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:47.878842 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4d1ee14d-94dc-4b39-995a-a0c21f356267 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:47.896323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1ee14d-94dc-4b39-995a-a0c21f356267 service nova] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:47.901790 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e1b4a-007a-473f-8aca-0b9aff2097a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '143c30b4201645279564e72ab3288adc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:47.902497 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e1b4a-007a-473f-8aca-0b9aff2097a3 service placement] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:47.902839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e1b4a-007a-473f-8aca-0b9aff2097a3 service placement] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:47.903198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e1b4a-007a-473f-8aca-0b9aff2097a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:47.905219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-147e1b4a-007a-473f-8aca-0b9aff2097a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:47.939411 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e1b4a-007a-473f-8aca-0b9aff2097a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:47.941820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e1b4a-007a-473f-8aca-0b9aff2097a3 service placement] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:47.960654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d46c87-37fa-4ff6-be3b-cde01faeafd2 service nova] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:47.962474 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e0d46c87-37fa-4ff6-be3b-cde01faeafd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:47.966057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1141/2291] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:31:47.993901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d46c87-37fa-4ff6-be3b-cde01faeafd2 service nova] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:47.998593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6608e-c08f-4430-b7de-16d8a519affa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:47.999250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6608e-c08f-4430-b7de-16d8a519affa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:47.999604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6608e-c08f-4430-b7de-16d8a519affa admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:47.999884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6608e-c08f-4430-b7de-16d8a519affa admin admin] PATH_INFO: `/v3/users/f643fd4ee6bc4ccfae45f3b8523ae46a` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:48.001093 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6608e-c08f-4430-b7de-16d8a519affa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f643fd4ee6bc4ccfae45f3b8523ae46a)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:48.003013 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6608e-c08f-4430-b7de-16d8a519affa admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:48.053973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-caa6608e-c08f-4430-b7de-16d8a519affa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f643fd4ee6bc4ccfae45f3b8523ae46a'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:48.066528 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-caa6608e-c08f-4430-b7de-16d8a519affa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f643fd4ee6bc4ccfae45f3b8523ae46a'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:48.073542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-caa6608e-c08f-4430-b7de-16d8a519affa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f643fd4ee6bc4ccfae45f3b8523ae46a'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:48.074341 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-caa6608e-c08f-4430-b7de-16d8a519affa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f643fd4ee6bc4ccfae45f3b8523ae46a'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:48.075008 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-caa6608e-c08f-4430-b7de-16d8a519affa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f643fd4ee6bc4ccfae45f3b8523ae46a'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:48.087883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1151/2292] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:31:47 2022] DELETE /identity/v3/users/f643fd4ee6bc4ccfae45f3b8523ae46a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:31:48.279835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e1b4a-007a-473f-8aca-0b9aff2097a3 service placement] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:48.291624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e1b4a-007a-473f-8aca-0b9aff2097a3 service placement] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:48.359662 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba42d90-a9b2-4211-9fe5-3f7a82ac80c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:48.360344 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba42d90-a9b2-4211-9fe5-3f7a82ac80c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:48.360611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba42d90-a9b2-4211-9fe5-3f7a82ac80c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:48.360890 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba42d90-a9b2-4211-9fe5-3f7a82ac80c9 admin admin] PATH_INFO: `/v3/projects/0fe60f1637744c3f8a25e83f48bfdf71` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:48.362080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba42d90-a9b2-4211-9fe5-3f7a82ac80c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fe60f1637744c3f8a25e83f48bfdf71)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:48.364019 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba42d90-a9b2-4211-9fe5-3f7a82ac80c9 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:48.411318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4ba42d90-a9b2-4211-9fe5-3f7a82ac80c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fe60f1637744c3f8a25e83f48bfdf71', 'request_id': 'req-4ba42d90-a9b2-4211-9fe5-3f7a82ac80c9'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:48.416957 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4ba42d90-a9b2-4211-9fe5-3f7a82ac80c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fe60f1637744c3f8a25e83f48bfdf71', 'request_id': 'req-4ba42d90-a9b2-4211-9fe5-3f7a82ac80c9'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:48.424368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4ba42d90-a9b2-4211-9fe5-3f7a82ac80c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fe60f1637744c3f8a25e83f48bfdf71', 'request_id': 'req-4ba42d90-a9b2-4211-9fe5-3f7a82ac80c9'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:48.438725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4ba42d90-a9b2-4211-9fe5-3f7a82ac80c9 admin admin] The token cache is being invalidate because project 0fe60f1637744c3f8a25e83f48bfdf71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:31:48.439011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4ba42d90-a9b2-4211-9fe5-3f7a82ac80c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:48.439669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4ba42d90-a9b2-4211-9fe5-3f7a82ac80c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:48.441210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1142/2293] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:31:48 2022] DELETE /identity/v3/projects/0fe60f1637744c3f8a25e83f48bfdf71 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:31:48.446407 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8462349-bd2c-47c3-9758-133590b61474 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:48.446698 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8462349-bd2c-47c3-9758-133590b61474 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:48.447006 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8462349-bd2c-47c3-9758-133590b61474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:48.469157 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-d8462349-bd2c-47c3-9758-133590b61474 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:31:48.470845 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d8462349-bd2c-47c3-9758-133590b61474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:48.543866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:31:48.565375 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1152/2294] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:31:48.573784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aa8a9-64f7-4d10-8610-2dec45ab1ee8 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:48.574072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aa8a9-64f7-4d10-8610-2dec45ab1ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:48.574354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607aa8a9-64f7-4d10-8610-2dec45ab1ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:48.601966 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-607aa8a9-64f7-4d10-8610-2dec45ab1ee8 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:31:48.603310 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-607aa8a9-64f7-4d10-8610-2dec45ab1ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:48.613674 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1143/2295] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:31:48.618155 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8462349-bd2c-47c3-9758-133590b61474 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:48.629992 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8462349-bd2c-47c3-9758-133590b61474 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:48.634185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8df78-e6af-4b9c-a2da-5db917b416ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:48.634997 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8df78-e6af-4b9c-a2da-5db917b416ac admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:48.635353 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8df78-e6af-4b9c-a2da-5db917b416ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:48.635850 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8df78-e6af-4b9c-a2da-5db917b416ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:48.636621 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8df78-e6af-4b9c-a2da-5db917b416ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:31:48.636994 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8df78-e6af-4b9c-a2da-5db917b416ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:48.638675 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f8df78-e6af-4b9c-a2da-5db917b416ac admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:48.653065 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1153/2296] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:31:48.657593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aa8a9-64f7-4d10-8610-2dec45ab1ee8 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:48.670779 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607aa8a9-64f7-4d10-8610-2dec45ab1ee8 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:48.678034 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab0fac-e77b-4389-87c8-a53b9c6f8ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:48.678666 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab0fac-e77b-4389-87c8-a53b9c6f8ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:48.678938 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab0fac-e77b-4389-87c8-a53b9c6f8ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:48.679201 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab0fac-e77b-4389-87c8-a53b9c6f8ac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:48.679830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab0fac-e77b-4389-87c8-a53b9c6f8ac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:48.681808 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab0fac-e77b-4389-87c8-a53b9c6f8ac4 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:48.712484 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1144/2297] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:31:48 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:31:48.716704 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8df78-e6af-4b9c-a2da-5db917b416ac admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:48.729355 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8df78-e6af-4b9c-a2da-5db917b416ac admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:48.790437 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc241d50-2588-4903-ab30-b8ab1f88424a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:48.791053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc241d50-2588-4903-ab30-b8ab1f88424a admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:48.791339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc241d50-2588-4903-ab30-b8ab1f88424a admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:48.791616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc241d50-2588-4903-ab30-b8ab1f88424a admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:48.792309 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc241d50-2588-4903-ab30-b8ab1f88424a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:48.794243 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc241d50-2588-4903-ab30-b8ab1f88424a admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:48.816806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1154/2298] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:31:48 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:31:48.821137 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab0fac-e77b-4389-87c8-a53b9c6f8ac4 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:48.831534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab0fac-e77b-4389-87c8-a53b9c6f8ac4 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:48.836782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7188ea-6059-4a20-b23e-11b47bf9fa17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:48.837370 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7188ea-6059-4a20-b23e-11b47bf9fa17 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:48.837646 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7188ea-6059-4a20-b23e-11b47bf9fa17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:48.837978 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7188ea-6059-4a20-b23e-11b47bf9fa17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:48.838565 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7188ea-6059-4a20-b23e-11b47bf9fa17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:48.840225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7188ea-6059-4a20-b23e-11b47bf9fa17 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:48.856635 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1145/2299] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:31:48.861083 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc241d50-2588-4903-ab30-b8ab1f88424a admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:48.871825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc241d50-2588-4903-ab30-b8ab1f88424a admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:48.876750 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3fbc4a-12c9-424f-988f-6b4613a9f051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:48.877511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fbc4a-12c9-424f-988f-6b4613a9f051 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:48.877658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fbc4a-12c9-424f-988f-6b4613a9f051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:48.877931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3fbc4a-12c9-424f-988f-6b4613a9f051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:48.878591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3fbc4a-12c9-424f-988f-6b4613a9f051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:48.880235 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3fbc4a-12c9-424f-988f-6b4613a9f051 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:48.891788 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1155/2300] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:31:48.896253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7188ea-6059-4a20-b23e-11b47bf9fa17 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:48.906732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7188ea-6059-4a20-b23e-11b47bf9fa17 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:48.911393 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde13eb8-4271-474e-8e0d-014fd394c2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:48.912204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde13eb8-4271-474e-8e0d-014fd394c2b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:48.912483 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde13eb8-4271-474e-8e0d-014fd394c2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:48.912753 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde13eb8-4271-474e-8e0d-014fd394c2b8 admin admin] PATH_INFO: `/v3/projects/c81753237bfb400393daa985c685e916/users/cfdaf6b43457427f8a295d00fb94ac9a/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:48.929454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde13eb8-4271-474e-8e0d-014fd394c2b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c81753237bfb400393daa985c685e916, user_id=cfdaf6b43457427f8a295d00fb94ac9a, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:48.932208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde13eb8-4271-474e-8e0d-014fd394c2b8 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:48.948918 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1146/2301] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:31:48 2022] PUT /identity/v3/projects/c81753237bfb400393daa985c685e916/users/cfdaf6b43457427f8a295d00fb94ac9a/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:31:48.961051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e821998-4a4d-4134-8a0b-7ff4b2197db4 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:48.961650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e821998-4a4d-4134-8a0b-7ff4b2197db4 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:48.961650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e821998-4a4d-4134-8a0b-7ff4b2197db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:49.004054 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-2e821998-4a4d-4134-8a0b-7ff4b2197db4 None None] MFA Rules not processed for user `cfdaf6b43457427f8a295d00fb94ac9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:31:49.005403 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-2e821998-4a4d-4134-8a0b-7ff4b2197db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:49.099126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1156/2302] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:31:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:31:49.104979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde13eb8-4271-474e-8e0d-014fd394c2b8 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:49.111052 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:31:49.174524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde13eb8-4271-474e-8e0d-014fd394c2b8 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:49.179780 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256be725-75b6-4f56-84c0-1770d1af383f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:49.180467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256be725-75b6-4f56-84c0-1770d1af383f admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:49.180744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256be725-75b6-4f56-84c0-1770d1af383f admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:49.181287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256be725-75b6-4f56-84c0-1770d1af383f admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:49.181929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256be725-75b6-4f56-84c0-1770d1af383f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:49.183862 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256be725-75b6-4f56-84c0-1770d1af383f admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:49.217916 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1147/2303] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:31:49 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:31:49.219421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e821998-4a4d-4134-8a0b-7ff4b2197db4 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:49.231702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e821998-4a4d-4134-8a0b-7ff4b2197db4 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:49.290272 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:31:49.309202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b8b4d-b7f8-4197-a1de-32bb7d819b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:49.309816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b8b4d-b7f8-4197-a1de-32bb7d819b85 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:49.310096 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b8b4d-b7f8-4197-a1de-32bb7d819b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:49.310368 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b8b4d-b7f8-4197-a1de-32bb7d819b85 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:49.311012 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b8b4d-b7f8-4197-a1de-32bb7d819b85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:49.313192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b8b4d-b7f8-4197-a1de-32bb7d819b85 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:49.338940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1157/2304] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:31:49 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:31:49.343340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256be725-75b6-4f56-84c0-1770d1af383f admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:49.357160 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256be725-75b6-4f56-84c0-1770d1af383f admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:49.362546 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c9c1a-799a-4aaf-b881-85bb7ddd3fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:49.363195 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c9c1a-799a-4aaf-b881-85bb7ddd3fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:49.363474 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c9c1a-799a-4aaf-b881-85bb7ddd3fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:49.363750 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c9c1a-799a-4aaf-b881-85bb7ddd3fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:49.364488 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c9c1a-799a-4aaf-b881-85bb7ddd3fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:49.366235 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c9c1a-799a-4aaf-b881-85bb7ddd3fb8 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:49.378871 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1148/2305] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:31:49.383503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b8b4d-b7f8-4197-a1de-32bb7d819b85 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:49.395070 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b8b4d-b7f8-4197-a1de-32bb7d819b85 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:49.399894 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb75f0-9316-4e69-96f8-b1fce34ec38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:49.400715 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb75f0-9316-4e69-96f8-b1fce34ec38d admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:49.401082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb75f0-9316-4e69-96f8-b1fce34ec38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:49.401426 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb75f0-9316-4e69-96f8-b1fce34ec38d admin admin] PATH_INFO: `/v3/projects/10a89a8f78104eee9a318b21e2cac893/users/98ecec104def4fdb81ce7632cfa0f46a/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:49.422204 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb75f0-9316-4e69-96f8-b1fce34ec38d admin admin] RBAC: Authorizing `identity:create_grant(project_id=10a89a8f78104eee9a318b21e2cac893, user_id=98ecec104def4fdb81ce7632cfa0f46a, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:49.429162 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb75f0-9316-4e69-96f8-b1fce34ec38d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:49.451553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1158/2306] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:31:49 2022] PUT /identity/v3/projects/10a89a8f78104eee9a318b21e2cac893/users/98ecec104def4fdb81ce7632cfa0f46a/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:31:49.455795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c9c1a-799a-4aaf-b881-85bb7ddd3fb8 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:49.466868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c9c1a-799a-4aaf-b881-85bb7ddd3fb8 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:49.537722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8164331-9b60-4c34-ae27-99fd9fede055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:49.538939 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8164331-9b60-4c34-ae27-99fd9fede055 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:49.539228 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8164331-9b60-4c34-ae27-99fd9fede055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:49.539683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8164331-9b60-4c34-ae27-99fd9fede055 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:49.540586 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8164331-9b60-4c34-ae27-99fd9fede055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:49.542218 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8164331-9b60-4c34-ae27-99fd9fede055 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:49.556300 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1149/2307] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:31:49.560821 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb75f0-9316-4e69-96f8-b1fce34ec38d admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:49.573621 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb75f0-9316-4e69-96f8-b1fce34ec38d admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:49.578563 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c697a-e4b7-44d8-92dc-6f958228689c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:49.579244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c697a-e4b7-44d8-92dc-6f958228689c admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:49.579533 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c697a-e4b7-44d8-92dc-6f958228689c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:49.579813 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c697a-e4b7-44d8-92dc-6f958228689c admin admin] PATH_INFO: `/v3/projects/10a89a8f78104eee9a318b21e2cac893/users/98ecec104def4fdb81ce7632cfa0f46a/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:49.581783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c697a-e4b7-44d8-92dc-6f958228689c admin admin] RBAC: Authorizing `identity:create_grant(project_id=10a89a8f78104eee9a318b21e2cac893, user_id=98ecec104def4fdb81ce7632cfa0f46a, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:49.584480 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c697a-e4b7-44d8-92dc-6f958228689c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:49.595618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1159/2308] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:31:49 2022] PUT /identity/v3/projects/10a89a8f78104eee9a318b21e2cac893/users/98ecec104def4fdb81ce7632cfa0f46a/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:31:49.606111 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1eba2-a884-4fbf-8438-2dac7d857294 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:49.606302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1eba2-a884-4fbf-8438-2dac7d857294 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:49.606582 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1eba2-a884-4fbf-8438-2dac7d857294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:49.647564 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-69d1eba2-a884-4fbf-8438-2dac7d857294 None None] MFA Rules not processed for user `98ecec104def4fdb81ce7632cfa0f46a`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:31:49.648911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-69d1eba2-a884-4fbf-8438-2dac7d857294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:49.751521 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:31:49.758847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1150/2309] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:31:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:31:49.772668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c697a-e4b7-44d8-92dc-6f958228689c admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:49.775950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1e0c697a-e4b7-44d8-92dc-6f958228689c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:49.846454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c697a-e4b7-44d8-92dc-6f958228689c admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:49.851414 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ca93b-b454-4599-b0d9-58f9ff8a2454 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:49.852085 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ca93b-b454-4599-b0d9-58f9ff8a2454 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:49.852927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ca93b-b454-4599-b0d9-58f9ff8a2454 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:49.853126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ca93b-b454-4599-b0d9-58f9ff8a2454 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:49.865137 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ca93b-b454-4599-b0d9-58f9ff8a2454 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:49.866851 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ca93b-b454-4599-b0d9-58f9ff8a2454 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:49.877514 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1160/2310] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:31:50.317547 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1eba2-a884-4fbf-8438-2dac7d857294 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:50.395014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1eba2-a884-4fbf-8438-2dac7d857294 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:50.400116 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224a997-3472-4165-b588-3b44c8a9fc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:50.400761 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224a997-3472-4165-b588-3b44c8a9fc7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:50.401040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224a997-3472-4165-b588-3b44c8a9fc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:50.401312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224a997-3472-4165-b588-3b44c8a9fc7e admin admin] PATH_INFO: `/v3/users/cfdaf6b43457427f8a295d00fb94ac9a` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:50.402545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224a997-3472-4165-b588-3b44c8a9fc7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfdaf6b43457427f8a295d00fb94ac9a)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:50.404584 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224a997-3472-4165-b588-3b44c8a9fc7e admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:50.461274 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6224a997-3472-4165-b588-3b44c8a9fc7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfdaf6b43457427f8a295d00fb94ac9a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:50.470214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6224a997-3472-4165-b588-3b44c8a9fc7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfdaf6b43457427f8a295d00fb94ac9a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:50.485119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6224a997-3472-4165-b588-3b44c8a9fc7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfdaf6b43457427f8a295d00fb94ac9a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:50.486594 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6224a997-3472-4165-b588-3b44c8a9fc7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfdaf6b43457427f8a295d00fb94ac9a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:50.494832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6224a997-3472-4165-b588-3b44c8a9fc7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfdaf6b43457427f8a295d00fb94ac9a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:50.501454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1151/2311] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:31:50 2022] DELETE /identity/v3/users/cfdaf6b43457427f8a295d00fb94ac9a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:31:50.506208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf94728-45f7-48b6-ada2-42d6e9fddb4b None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:50.506881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf94728-45f7-48b6-ada2-42d6e9fddb4b None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:50.506881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf94728-45f7-48b6-ada2-42d6e9fddb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:50.528064 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-8cf94728-45f7-48b6-ada2-42d6e9fddb4b None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:31:50.529715 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8cf94728-45f7-48b6-ada2-42d6e9fddb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:50.620820 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1161/2312] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:31:50.636209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224a997-3472-4165-b588-3b44c8a9fc7e admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:50.638042 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-6224a997-3472-4165-b588-3b44c8a9fc7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:50.732268 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224a997-3472-4165-b588-3b44c8a9fc7e admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:50.736367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fc896-420b-4f03-8172-c2e150e01cf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:50.737112 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fc896-420b-4f03-8172-c2e150e01cf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:50.737454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fc896-420b-4f03-8172-c2e150e01cf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:50.737793 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9fc896-420b-4f03-8172-c2e150e01cf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:50.759960 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fc896-420b-4f03-8172-c2e150e01cf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:50.761759 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9fc896-420b-4f03-8172-c2e150e01cf0 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:50.776230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1152/2313] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:31:51.430640 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf94728-45f7-48b6-ada2-42d6e9fddb4b None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:51.433721 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8cf94728-45f7-48b6-ada2-42d6e9fddb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:51.452435 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf94728-45f7-48b6-ada2-42d6e9fddb4b None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:51.458380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3f49a-6359-4483-95d4-35d23267c97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:51.459235 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3f49a-6359-4483-95d4-35d23267c97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:51.459670 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3f49a-6359-4483-95d4-35d23267c97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:51.460107 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3f49a-6359-4483-95d4-35d23267c97b admin admin] PATH_INFO: `/v3/projects/c81753237bfb400393daa985c685e916` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:51.461674 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c3f49a-6359-4483-95d4-35d23267c97b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c81753237bfb400393daa985c685e916)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:51.464190 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c3f49a-6359-4483-95d4-35d23267c97b admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:51.515796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:31:51.525194 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-67c3f49a-6359-4483-95d4-35d23267c97b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c81753237bfb400393daa985c685e916', 'request_id': 'req-67c3f49a-6359-4483-95d4-35d23267c97b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:51.532609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-67c3f49a-6359-4483-95d4-35d23267c97b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c81753237bfb400393daa985c685e916', 'request_id': 'req-67c3f49a-6359-4483-95d4-35d23267c97b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:51.543253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-67c3f49a-6359-4483-95d4-35d23267c97b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c81753237bfb400393daa985c685e916', 'request_id': 'req-67c3f49a-6359-4483-95d4-35d23267c97b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:51.557396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-67c3f49a-6359-4483-95d4-35d23267c97b admin admin] The token cache is being invalidate because project c81753237bfb400393daa985c685e916 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:31:51.557700 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-67c3f49a-6359-4483-95d4-35d23267c97b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:51.558390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-67c3f49a-6359-4483-95d4-35d23267c97b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:51.560015 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1162/2314] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:31:51 2022] DELETE /identity/v3/projects/c81753237bfb400393daa985c685e916 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:31:51.564319 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fc896-420b-4f03-8172-c2e150e01cf0 service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:51.575011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9fc896-420b-4f03-8172-c2e150e01cf0 service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:51.593746 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:31:51.641410 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a75318-8f0c-4a4d-bd99-5b18584dcb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:51.642545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a75318-8f0c-4a4d-bd99-5b18584dcb2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:51.642545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a75318-8f0c-4a4d-bd99-5b18584dcb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:51.642875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a75318-8f0c-4a4d-bd99-5b18584dcb2c admin admin] PATH_INFO: `/v3/users/98ecec104def4fdb81ce7632cfa0f46a` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:51.644285 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a75318-8f0c-4a4d-bd99-5b18584dcb2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ecec104def4fdb81ce7632cfa0f46a)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:51.646655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a75318-8f0c-4a4d-bd99-5b18584dcb2c admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:51.700159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e0a75318-8f0c-4a4d-bd99-5b18584dcb2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ecec104def4fdb81ce7632cfa0f46a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:51.708048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e0a75318-8f0c-4a4d-bd99-5b18584dcb2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ecec104def4fdb81ce7632cfa0f46a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:51.720124 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e0a75318-8f0c-4a4d-bd99-5b18584dcb2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ecec104def4fdb81ce7632cfa0f46a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:51.720872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e0a75318-8f0c-4a4d-bd99-5b18584dcb2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ecec104def4fdb81ce7632cfa0f46a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:51.729663 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e0a75318-8f0c-4a4d-bd99-5b18584dcb2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ecec104def4fdb81ce7632cfa0f46a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:51.736637 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1153/2315] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:31:51 2022] DELETE /identity/v3/users/98ecec104def4fdb81ce7632cfa0f46a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:31:52.027554 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3f49a-6359-4483-95d4-35d23267c97b admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:52.029204 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-67c3f49a-6359-4483-95d4-35d23267c97b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:52.045081 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3f49a-6359-4483-95d4-35d23267c97b admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:52.128236 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc49b4c-eb22-4670-8f3d-49ac95282e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:52.128886 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc49b4c-eb22-4670-8f3d-49ac95282e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:52.129164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc49b4c-eb22-4670-8f3d-49ac95282e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:52.129458 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc49b4c-eb22-4670-8f3d-49ac95282e1e admin admin] PATH_INFO: `/v3/projects/10a89a8f78104eee9a318b21e2cac893` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:52.130563 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc49b4c-eb22-4670-8f3d-49ac95282e1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=10a89a8f78104eee9a318b21e2cac893)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:52.132563 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc49b4c-eb22-4670-8f3d-49ac95282e1e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:52.179589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-cfc49b4c-eb22-4670-8f3d-49ac95282e1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10a89a8f78104eee9a318b21e2cac893', 'request_id': 'req-cfc49b4c-eb22-4670-8f3d-49ac95282e1e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:52.185849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-cfc49b4c-eb22-4670-8f3d-49ac95282e1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10a89a8f78104eee9a318b21e2cac893', 'request_id': 'req-cfc49b4c-eb22-4670-8f3d-49ac95282e1e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:52.196416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-cfc49b4c-eb22-4670-8f3d-49ac95282e1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10a89a8f78104eee9a318b21e2cac893', 'request_id': 'req-cfc49b4c-eb22-4670-8f3d-49ac95282e1e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:52.209620 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-cfc49b4c-eb22-4670-8f3d-49ac95282e1e admin admin] The token cache is being invalidate because project 10a89a8f78104eee9a318b21e2cac893 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:31:52.209897 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-cfc49b4c-eb22-4670-8f3d-49ac95282e1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:52.211085 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-cfc49b4c-eb22-4670-8f3d-49ac95282e1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:31:52.212890 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1163/2316] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:31:52 2022] DELETE /identity/v3/projects/10a89a8f78104eee9a318b21e2cac893 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:31:52.217577 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d965861-d72b-4d9d-a0f6-c19f351f976a None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:52.217943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d965861-d72b-4d9d-a0f6-c19f351f976a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:52.218307 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d965861-d72b-4d9d-a0f6-c19f351f976a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:52.237604 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-2d965861-d72b-4d9d-a0f6-c19f351f976a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:31:52.238817 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-2d965861-d72b-4d9d-a0f6-c19f351f976a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:52.302411 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1154/2317] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:31:52.310729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa8703-f7a5-4481-b3c6-968568ae8563 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:52.311014 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa8703-f7a5-4481-b3c6-968568ae8563 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:52.311285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa8703-f7a5-4481-b3c6-968568ae8563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:52.331736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-cdfa8703-f7a5-4481-b3c6-968568ae8563 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:31:52.332960 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-cdfa8703-f7a5-4481-b3c6-968568ae8563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:52.341420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1164/2318] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:31:52.345527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d965861-d72b-4d9d-a0f6-c19f351f976a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:52.356094 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d965861-d72b-4d9d-a0f6-c19f351f976a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:52.362254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23b9282-4f06-4ad3-b556-082d4cfececa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:52.362892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b9282-4f06-4ad3-b556-082d4cfececa admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:52.363171 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b9282-4f06-4ad3-b556-082d4cfececa admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:52.363592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b9282-4f06-4ad3-b556-082d4cfececa admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:52.364137 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23b9282-4f06-4ad3-b556-082d4cfececa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:31:52.364444 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23b9282-4f06-4ad3-b556-082d4cfececa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:52.366063 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23b9282-4f06-4ad3-b556-082d4cfececa admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:52.380085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1155/2319] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:31:52.385193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa8703-f7a5-4481-b3c6-968568ae8563 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:52.410843 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa8703-f7a5-4481-b3c6-968568ae8563 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:52.420844 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d16e98-ea15-4023-b613-031f8b2e69b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:52.422171 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d16e98-ea15-4023-b613-031f8b2e69b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:52.422675 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d16e98-ea15-4023-b613-031f8b2e69b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:52.423883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d16e98-ea15-4023-b613-031f8b2e69b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:52.424944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d16e98-ea15-4023-b613-031f8b2e69b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:52.429692 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d16e98-ea15-4023-b613-031f8b2e69b9 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:52.467061 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1165/2320] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:31:52 2022] POST /identity/v3/projects => generated 409 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:31:52.471682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23b9282-4f06-4ad3-b556-082d4cfececa admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:52.486136 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23b9282-4f06-4ad3-b556-082d4cfececa admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:52.515816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:31:52.552941 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85676662-ebcf-47dc-beec-fa36b10dd96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:52.553745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85676662-ebcf-47dc-beec-fa36b10dd96a admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:52.554066 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85676662-ebcf-47dc-beec-fa36b10dd96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:52.554342 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85676662-ebcf-47dc-beec-fa36b10dd96a admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:52.555040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85676662-ebcf-47dc-beec-fa36b10dd96a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:52.557387 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85676662-ebcf-47dc-beec-fa36b10dd96a admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:52.579764 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1156/2321] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:31:52 2022] POST /identity/v3/users => generated 305 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:31:52.584018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d16e98-ea15-4023-b613-031f8b2e69b9 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:52.594829 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d16e98-ea15-4023-b613-031f8b2e69b9 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:52.600969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084b7a2-bfb1-45a2-855c-debd5158bde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:52.601578 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084b7a2-bfb1-45a2-855c-debd5158bde9 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:52.601853 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084b7a2-bfb1-45a2-855c-debd5158bde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:52.602119 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084b7a2-bfb1-45a2-855c-debd5158bde9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:52.602768 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084b7a2-bfb1-45a2-855c-debd5158bde9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:52.604430 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084b7a2-bfb1-45a2-855c-debd5158bde9 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:52.614868 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1166/2322] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:31:52.619100 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85676662-ebcf-47dc-beec-fa36b10dd96a admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:52.629947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85676662-ebcf-47dc-beec-fa36b10dd96a admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:52.636113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19ce0a-0592-48bc-9b4b-0ed27fbdb7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:52.636739 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19ce0a-0592-48bc-9b4b-0ed27fbdb7ee admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:52.637022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19ce0a-0592-48bc-9b4b-0ed27fbdb7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:52.637286 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19ce0a-0592-48bc-9b4b-0ed27fbdb7ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:52.637928 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19ce0a-0592-48bc-9b4b-0ed27fbdb7ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:52.639547 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19ce0a-0592-48bc-9b4b-0ed27fbdb7ee admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:52.650482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1157/2323] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:31:52.654959 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084b7a2-bfb1-45a2-855c-debd5158bde9 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:52.665384 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084b7a2-bfb1-45a2-855c-debd5158bde9 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:52.671060 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48085d4f-a041-4865-a673-647750efa08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:52.671731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48085d4f-a041-4865-a673-647750efa08d admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:52.672016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48085d4f-a041-4865-a673-647750efa08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:52.672312 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48085d4f-a041-4865-a673-647750efa08d admin admin] PATH_INFO: `/v3/projects/176d66fff15c4de4bee460743dcd02d0/users/0ec968abea0049f387095969b4882c6c/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:52.688209 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48085d4f-a041-4865-a673-647750efa08d admin admin] RBAC: Authorizing `identity:create_grant(project_id=176d66fff15c4de4bee460743dcd02d0, user_id=0ec968abea0049f387095969b4882c6c, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:52.690790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48085d4f-a041-4865-a673-647750efa08d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:52.692992 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:31:52.708505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1167/2324] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:31:52 2022] PUT /identity/v3/projects/176d66fff15c4de4bee460743dcd02d0/users/0ec968abea0049f387095969b4882c6c/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:31:52.720285 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d314ade-b68b-4480-91b0-925cfff0f288 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:52.720762 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d314ade-b68b-4480-91b0-925cfff0f288 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:52.721107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d314ade-b68b-4480-91b0-925cfff0f288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:52.758617 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-8d314ade-b68b-4480-91b0-925cfff0f288 None None] MFA Rules not processed for user `0ec968abea0049f387095969b4882c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:31:52.759884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8d314ade-b68b-4480-91b0-925cfff0f288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:52.851398 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1158/2325] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:31:52 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:31:52.856769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48085d4f-a041-4865-a673-647750efa08d admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:52.925354 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48085d4f-a041-4865-a673-647750efa08d admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:52.931031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a28805-b5a5-4bd1-8bea-2288855370de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:52.931650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a28805-b5a5-4bd1-8bea-2288855370de admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:52.931949 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a28805-b5a5-4bd1-8bea-2288855370de admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:52.932251 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a28805-b5a5-4bd1-8bea-2288855370de admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:52.932997 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a28805-b5a5-4bd1-8bea-2288855370de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:52.935020 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a28805-b5a5-4bd1-8bea-2288855370de admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:52.967961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1168/2326] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:31:52 2022] POST /identity/v3/projects => generated 409 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:31:52.972468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d314ade-b68b-4480-91b0-925cfff0f288 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:52.983657 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d314ade-b68b-4480-91b0-925cfff0f288 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:53.060280 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4c337-ae1f-43ac-b31f-cfca88a57fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:53.061271 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4c337-ae1f-43ac-b31f-cfca88a57fbc admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:53.061752 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4c337-ae1f-43ac-b31f-cfca88a57fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:53.062153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4c337-ae1f-43ac-b31f-cfca88a57fbc admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:53.062931 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4c337-ae1f-43ac-b31f-cfca88a57fbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:53.065107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4c337-ae1f-43ac-b31f-cfca88a57fbc admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:53.093568 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1159/2327] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:31:52 2022] POST /identity/v3/users => generated 311 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:31:53.094865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a28805-b5a5-4bd1-8bea-2288855370de admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:53.105712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a28805-b5a5-4bd1-8bea-2288855370de admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:53.110824 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9c49f-5c73-4824-94dc-339bc82af6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:53.111820 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9c49f-5c73-4824-94dc-339bc82af6f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:53.112331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9c49f-5c73-4824-94dc-339bc82af6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:53.112793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad9c49f-5c73-4824-94dc-339bc82af6f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:53.113642 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9c49f-5c73-4824-94dc-339bc82af6f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:53.115337 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad9c49f-5c73-4824-94dc-339bc82af6f5 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:53.126726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1169/2328] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:31:53.131202 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4c337-ae1f-43ac-b31f-cfca88a57fbc admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:53.141679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4c337-ae1f-43ac-b31f-cfca88a57fbc admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:53.146495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee49214-b997-471d-9935-8e135848c33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:53.147172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee49214-b997-471d-9935-8e135848c33e admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:53.147453 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee49214-b997-471d-9935-8e135848c33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:53.147744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee49214-b997-471d-9935-8e135848c33e admin admin] PATH_INFO: `/v3/projects/c52ed6a2a4d34192bcf12b314dc8b403/users/9ab6bfb4b61944cb856429b52f4d3f38/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:53.163959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee49214-b997-471d-9935-8e135848c33e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c52ed6a2a4d34192bcf12b314dc8b403, user_id=9ab6bfb4b61944cb856429b52f4d3f38, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:53.166635 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee49214-b997-471d-9935-8e135848c33e admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:53.177509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:31:53.182948 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1160/2329] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:31:53 2022] PUT /identity/v3/projects/c52ed6a2a4d34192bcf12b314dc8b403/users/9ab6bfb4b61944cb856429b52f4d3f38/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:31:53.187143 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9c49f-5c73-4824-94dc-339bc82af6f5 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:53.197708 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad9c49f-5c73-4824-94dc-339bc82af6f5 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:53.259580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afd31e-4c67-43e9-b315-db303229845d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:53.260243 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afd31e-4c67-43e9-b315-db303229845d admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:53.260523 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afd31e-4c67-43e9-b315-db303229845d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:53.260791 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afd31e-4c67-43e9-b315-db303229845d admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:53.261437 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9afd31e-4c67-43e9-b315-db303229845d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:53.263114 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9afd31e-4c67-43e9-b315-db303229845d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:53.277497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1170/2330] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:31:53.278764 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee49214-b997-471d-9935-8e135848c33e admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:53.289291 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee49214-b997-471d-9935-8e135848c33e admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:53.294072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22806fb9-fa94-44b8-a96a-5fe0baefd76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:53.294734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22806fb9-fa94-44b8-a96a-5fe0baefd76e admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:53.295007 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22806fb9-fa94-44b8-a96a-5fe0baefd76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:53.295639 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22806fb9-fa94-44b8-a96a-5fe0baefd76e admin admin] PATH_INFO: `/v3/projects/c52ed6a2a4d34192bcf12b314dc8b403/users/9ab6bfb4b61944cb856429b52f4d3f38/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:53.297562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22806fb9-fa94-44b8-a96a-5fe0baefd76e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c52ed6a2a4d34192bcf12b314dc8b403, user_id=9ab6bfb4b61944cb856429b52f4d3f38, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:53.300250 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22806fb9-fa94-44b8-a96a-5fe0baefd76e admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:53.311179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1161/2331] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:31:53 2022] PUT /identity/v3/projects/c52ed6a2a4d34192bcf12b314dc8b403/users/9ab6bfb4b61944cb856429b52f4d3f38/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:31:53.321991 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1cfd9-099e-4b67-bdec-358b062ab6d8 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:53.322279 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1cfd9-099e-4b67-bdec-358b062ab6d8 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:53.322531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1cfd9-099e-4b67-bdec-358b062ab6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:53.353769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-eaf1cfd9-099e-4b67-bdec-358b062ab6d8 None None] MFA Rules not processed for user `9ab6bfb4b61944cb856429b52f4d3f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:31:53.355053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-eaf1cfd9-099e-4b67-bdec-358b062ab6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:53.405725 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:31:53.427409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1171/2332] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:31:53 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:31:53.438716 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22806fb9-fa94-44b8-a96a-5fe0baefd76e admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:53.440561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-22806fb9-fa94-44b8-a96a-5fe0baefd76e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:31:53.500680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22806fb9-fa94-44b8-a96a-5fe0baefd76e admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:53.504360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178faca-6406-421b-ab4c-897dd2dd9b8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:53.504979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178faca-6406-421b-ab4c-897dd2dd9b8c service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:53.505258 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178faca-6406-421b-ab4c-897dd2dd9b8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:53.505547 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178faca-6406-421b-ab4c-897dd2dd9b8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:53.515021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178faca-6406-421b-ab4c-897dd2dd9b8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:53.516545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178faca-6406-421b-ab4c-897dd2dd9b8c service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:53.525921 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1162/2333] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:53 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:31:53.688592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1cfd9-099e-4b67-bdec-358b062ab6d8 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:31:53.700160 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1cfd9-099e-4b67-bdec-358b062ab6d8 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:31:53.704780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae564f3-9e4a-456d-a402-08b8cf3dfbf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:31:53.705366 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae564f3-9e4a-456d-a402-08b8cf3dfbf7 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:31:53.705650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae564f3-9e4a-456d-a402-08b8cf3dfbf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:31:53.705919 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae564f3-9e4a-456d-a402-08b8cf3dfbf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:31:53.765618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae564f3-9e4a-456d-a402-08b8cf3dfbf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:31:53.766973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae564f3-9e4a-456d-a402-08b8cf3dfbf7 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:31:53.778790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1172/2334] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:31:53 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:01.234957 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178faca-6406-421b-ab4c-897dd2dd9b8c service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:01.312279 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178faca-6406-421b-ab4c-897dd2dd9b8c service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:01.317237 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da30b1-4707-4088-89ac-41e863e3d8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:01.317872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da30b1-4707-4088-89ac-41e863e3d8f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:01.318152 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da30b1-4707-4088-89ac-41e863e3d8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:01.318444 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da30b1-4707-4088-89ac-41e863e3d8f8 admin admin] PATH_INFO: `/v3/users/0ec968abea0049f387095969b4882c6c` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:01.319681 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da30b1-4707-4088-89ac-41e863e3d8f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ec968abea0049f387095969b4882c6c)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:01.321675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da30b1-4707-4088-89ac-41e863e3d8f8 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:01.370794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-c2da30b1-4707-4088-89ac-41e863e3d8f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ec968abea0049f387095969b4882c6c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:01.377659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-c2da30b1-4707-4088-89ac-41e863e3d8f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ec968abea0049f387095969b4882c6c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:01.390255 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-c2da30b1-4707-4088-89ac-41e863e3d8f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ec968abea0049f387095969b4882c6c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:01.390941 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-c2da30b1-4707-4088-89ac-41e863e3d8f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ec968abea0049f387095969b4882c6c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:01.400265 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-c2da30b1-4707-4088-89ac-41e863e3d8f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ec968abea0049f387095969b4882c6c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:01.406765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1163/2335] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:01 2022] DELETE /identity/v3/users/0ec968abea0049f387095969b4882c6c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:32:01.411498 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11927d5b-b2b3-4ca0-bf66-e7d875d75433 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:01.412039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11927d5b-b2b3-4ca0-bf66-e7d875d75433 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:01.412154 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11927d5b-b2b3-4ca0-bf66-e7d875d75433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:01.432037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-11927d5b-b2b3-4ca0-bf66-e7d875d75433 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:01.433266 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-11927d5b-b2b3-4ca0-bf66-e7d875d75433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:01.496133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1173/2336] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:01.506220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da30b1-4707-4088-89ac-41e863e3d8f8 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:01.507688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c2da30b1-4707-4088-89ac-41e863e3d8f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:01.518856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:01.572731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da30b1-4707-4088-89ac-41e863e3d8f8 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:01.576481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d5f66-78f6-4648-85f3-994b8b010f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:01.577166 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d5f66-78f6-4648-85f3-994b8b010f90 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:01.577443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d5f66-78f6-4648-85f3-994b8b010f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:01.577715 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d5f66-78f6-4648-85f3-994b8b010f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:01.587604 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d5f66-78f6-4648-85f3-994b8b010f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:01.589101 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d5f66-78f6-4648-85f3-994b8b010f90 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:01.599493 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1164/2337] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:01.896835 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11927d5b-b2b3-4ca0-bf66-e7d875d75433 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:01.898432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-11927d5b-b2b3-4ca0-bf66-e7d875d75433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:01.910734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11927d5b-b2b3-4ca0-bf66-e7d875d75433 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:01.915773 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bbd61-396c-4146-abae-6ba107e6d550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:01.916556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bbd61-396c-4146-abae-6ba107e6d550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:01.916813 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bbd61-396c-4146-abae-6ba107e6d550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:01.917119 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bbd61-396c-4146-abae-6ba107e6d550 admin admin] PATH_INFO: `/v3/projects/176d66fff15c4de4bee460743dcd02d0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:01.918315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bbd61-396c-4146-abae-6ba107e6d550 admin admin] RBAC: Authorizing `identity:delete_project(project_id=176d66fff15c4de4bee460743dcd02d0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:01.920661 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bbd61-396c-4146-abae-6ba107e6d550 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:01.971949 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-fc9bbd61-396c-4146-abae-6ba107e6d550 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '176d66fff15c4de4bee460743dcd02d0', 'request_id': 'req-fc9bbd61-396c-4146-abae-6ba107e6d550'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:01.978536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-fc9bbd61-396c-4146-abae-6ba107e6d550 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '176d66fff15c4de4bee460743dcd02d0', 'request_id': 'req-fc9bbd61-396c-4146-abae-6ba107e6d550'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:01.986601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-fc9bbd61-396c-4146-abae-6ba107e6d550 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '176d66fff15c4de4bee460743dcd02d0', 'request_id': 'req-fc9bbd61-396c-4146-abae-6ba107e6d550'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:01.997826 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-fc9bbd61-396c-4146-abae-6ba107e6d550 admin admin] The token cache is being invalidate because project 176d66fff15c4de4bee460743dcd02d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:01.999036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-fc9bbd61-396c-4146-abae-6ba107e6d550 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:01.999740 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-fc9bbd61-396c-4146-abae-6ba107e6d550 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:02.001520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1174/2338] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:01 2022] DELETE /identity/v3/projects/176d66fff15c4de4bee460743dcd02d0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:32:02.005575 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d5f66-78f6-4648-85f3-994b8b010f90 service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:02.017005 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d5f66-78f6-4648-85f3-994b8b010f90 service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:02.081633 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada67a7-3812-4cf5-b706-a3143076c051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:02.082284 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada67a7-3812-4cf5-b706-a3143076c051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:02.082554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada67a7-3812-4cf5-b706-a3143076c051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:02.082812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada67a7-3812-4cf5-b706-a3143076c051 admin admin] PATH_INFO: `/v3/users/9ab6bfb4b61944cb856429b52f4d3f38` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:02.084001 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ada67a7-3812-4cf5-b706-a3143076c051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab6bfb4b61944cb856429b52f4d3f38)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:02.085933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ada67a7-3812-4cf5-b706-a3143076c051 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:02.134273 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7ada67a7-3812-4cf5-b706-a3143076c051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab6bfb4b61944cb856429b52f4d3f38'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:02.146018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7ada67a7-3812-4cf5-b706-a3143076c051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab6bfb4b61944cb856429b52f4d3f38'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:02.159456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7ada67a7-3812-4cf5-b706-a3143076c051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab6bfb4b61944cb856429b52f4d3f38'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:02.160367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7ada67a7-3812-4cf5-b706-a3143076c051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab6bfb4b61944cb856429b52f4d3f38'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:02.171322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7ada67a7-3812-4cf5-b706-a3143076c051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab6bfb4b61944cb856429b52f4d3f38'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:02.178887 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1165/2339] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:02 2022] DELETE /identity/v3/users/9ab6bfb4b61944cb856429b52f4d3f38 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:02.490433 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bbd61-396c-4146-abae-6ba107e6d550 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:02.492182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-fc9bbd61-396c-4146-abae-6ba107e6d550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:02.507481 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bbd61-396c-4146-abae-6ba107e6d550 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:02.516644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:02.570791 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271732b6-99cf-4fe7-87a5-580f625239b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:02.571434 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271732b6-99cf-4fe7-87a5-580f625239b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:02.571714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271732b6-99cf-4fe7-87a5-580f625239b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:02.571985 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271732b6-99cf-4fe7-87a5-580f625239b5 admin admin] PATH_INFO: `/v3/projects/c52ed6a2a4d34192bcf12b314dc8b403` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:02.573256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271732b6-99cf-4fe7-87a5-580f625239b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c52ed6a2a4d34192bcf12b314dc8b403)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:02.575160 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271732b6-99cf-4fe7-87a5-580f625239b5 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:02.625683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-271732b6-99cf-4fe7-87a5-580f625239b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c52ed6a2a4d34192bcf12b314dc8b403', 'request_id': 'req-271732b6-99cf-4fe7-87a5-580f625239b5'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:02.632393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-271732b6-99cf-4fe7-87a5-580f625239b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c52ed6a2a4d34192bcf12b314dc8b403', 'request_id': 'req-271732b6-99cf-4fe7-87a5-580f625239b5'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:02.641863 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-271732b6-99cf-4fe7-87a5-580f625239b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c52ed6a2a4d34192bcf12b314dc8b403', 'request_id': 'req-271732b6-99cf-4fe7-87a5-580f625239b5'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:02.652451 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-271732b6-99cf-4fe7-87a5-580f625239b5 admin admin] The token cache is being invalidate because project c52ed6a2a4d34192bcf12b314dc8b403 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:02.652738 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-271732b6-99cf-4fe7-87a5-580f625239b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:02.653359 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-271732b6-99cf-4fe7-87a5-580f625239b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:02.654985 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1175/2340] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:02 2022] DELETE /identity/v3/projects/c52ed6a2a4d34192bcf12b314dc8b403 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:02.660127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079ae64-8f10-4961-bbe9-6d1e5e7d7771 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:02.660724 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079ae64-8f10-4961-bbe9-6d1e5e7d7771 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:02.660724 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079ae64-8f10-4961-bbe9-6d1e5e7d7771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:02.680889 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-b079ae64-8f10-4961-bbe9-6d1e5e7d7771 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:02.682215 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b079ae64-8f10-4961-bbe9-6d1e5e7d7771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:02.711384 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:02.747002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1166/2341] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:02.755266 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f606b-3c27-4af0-bb6a-9c99e1199335 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:02.755544 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f606b-3c27-4af0-bb6a-9c99e1199335 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:02.755827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f606b-3c27-4af0-bb6a-9c99e1199335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:02.776982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-f08f606b-3c27-4af0-bb6a-9c99e1199335 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:02.778226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f08f606b-3c27-4af0-bb6a-9c99e1199335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:02.787326 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1176/2342] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:02.791778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079ae64-8f10-4961-bbe9-6d1e5e7d7771 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:02.802552 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079ae64-8f10-4961-bbe9-6d1e5e7d7771 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:02.807601 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea1c18-b3ee-4a2d-9897-fba885468db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:02.808278 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea1c18-b3ee-4a2d-9897-fba885468db5 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:02.808607 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea1c18-b3ee-4a2d-9897-fba885468db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:02.808896 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea1c18-b3ee-4a2d-9897-fba885468db5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:02.809579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea1c18-b3ee-4a2d-9897-fba885468db5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:32:02.809878 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea1c18-b3ee-4a2d-9897-fba885468db5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:02.811517 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea1c18-b3ee-4a2d-9897-fba885468db5 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:02.824196 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1167/2343] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:32:02.828701 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f606b-3c27-4af0-bb6a-9c99e1199335 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:02.838335 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f606b-3c27-4af0-bb6a-9c99e1199335 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:02.843670 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e6f9b-a9b3-414a-ac0e-d7c299e1f0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:02.844298 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e6f9b-a9b3-414a-ac0e-d7c299e1f0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:02.844610 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e6f9b-a9b3-414a-ac0e-d7c299e1f0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:02.844889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e6f9b-a9b3-414a-ac0e-d7c299e1f0b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:02.845519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e6f9b-a9b3-414a-ac0e-d7c299e1f0b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:02.847417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e6f9b-a9b3-414a-ac0e-d7c299e1f0b4 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:02.876892 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1177/2344] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:02 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:02.881249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea1c18-b3ee-4a2d-9897-fba885468db5 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:02.891720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea1c18-b3ee-4a2d-9897-fba885468db5 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:02.949106 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f358f3-47af-4c4e-9aab-b43fe8980471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:02.949733 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f358f3-47af-4c4e-9aab-b43fe8980471 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:02.950017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f358f3-47af-4c4e-9aab-b43fe8980471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:02.950295 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f358f3-47af-4c4e-9aab-b43fe8980471 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:02.950918 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f358f3-47af-4c4e-9aab-b43fe8980471 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:02.952917 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f358f3-47af-4c4e-9aab-b43fe8980471 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:02.975966 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1168/2345] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:02 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:02.980267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e6f9b-a9b3-414a-ac0e-d7c299e1f0b4 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:02.991060 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e6f9b-a9b3-414a-ac0e-d7c299e1f0b4 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:02.996324 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f5201-eaa2-49f0-b59e-49eb02f5489c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:02.996938 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f5201-eaa2-49f0-b59e-49eb02f5489c admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:02.997217 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f5201-eaa2-49f0-b59e-49eb02f5489c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:02.997491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f5201-eaa2-49f0-b59e-49eb02f5489c admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:02.998149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f5201-eaa2-49f0-b59e-49eb02f5489c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:02.999819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f5201-eaa2-49f0-b59e-49eb02f5489c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:03.010715 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1178/2346] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:03.014717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f358f3-47af-4c4e-9aab-b43fe8980471 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:03.025879 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f358f3-47af-4c4e-9aab-b43fe8980471 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:03.032176 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ff856-3377-456c-a4fd-d454a2e2ad84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:03.032728 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ff856-3377-456c-a4fd-d454a2e2ad84 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:03.033007 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ff856-3377-456c-a4fd-d454a2e2ad84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:03.033275 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ff856-3377-456c-a4fd-d454a2e2ad84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:03.033921 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ff856-3377-456c-a4fd-d454a2e2ad84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:03.036074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ff856-3377-456c-a4fd-d454a2e2ad84 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:03.049135 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1169/2347] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:03.053880 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f5201-eaa2-49f0-b59e-49eb02f5489c admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:03.064538 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f5201-eaa2-49f0-b59e-49eb02f5489c admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:03.069633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8a8e0-c4e3-445c-84cb-7e46197e1dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:03.070283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8a8e0-c4e3-445c-84cb-7e46197e1dd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:03.070567 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8a8e0-c4e3-445c-84cb-7e46197e1dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:03.070841 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8a8e0-c4e3-445c-84cb-7e46197e1dd6 admin admin] PATH_INFO: `/v3/projects/a534f5b4d1d743cba1b7222fa173c73d/users/bad9100b16964faea77f1ad9e5eca214/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:03.085338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8a8e0-c4e3-445c-84cb-7e46197e1dd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a534f5b4d1d743cba1b7222fa173c73d, user_id=bad9100b16964faea77f1ad9e5eca214, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:03.087981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8a8e0-c4e3-445c-84cb-7e46197e1dd6 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:03.102720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1179/2348] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:03 2022] PUT /identity/v3/projects/a534f5b4d1d743cba1b7222fa173c73d/users/bad9100b16964faea77f1ad9e5eca214/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:03.115186 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cee46-a0eb-47bf-99e8-41750a81194e None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:03.115440 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cee46-a0eb-47bf-99e8-41750a81194e None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:03.115734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cee46-a0eb-47bf-99e8-41750a81194e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:03.145992 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-3d4cee46-a0eb-47bf-99e8-41750a81194e None None] MFA Rules not processed for user `bad9100b16964faea77f1ad9e5eca214`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:03.147806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3d4cee46-a0eb-47bf-99e8-41750a81194e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:03.220513 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1170/2349] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:03.225301 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8a8e0-c4e3-445c-84cb-7e46197e1dd6 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:03.269120 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:03.291322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8a8e0-c4e3-445c-84cb-7e46197e1dd6 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:03.296317 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfbab0d-0a44-42d7-9a1e-8cf2b1fbd0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:03.296926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbab0d-0a44-42d7-9a1e-8cf2b1fbd0a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:03.297205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbab0d-0a44-42d7-9a1e-8cf2b1fbd0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:03.297477 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfbab0d-0a44-42d7-9a1e-8cf2b1fbd0a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:03.298224 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfbab0d-0a44-42d7-9a1e-8cf2b1fbd0a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:03.300127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfbab0d-0a44-42d7-9a1e-8cf2b1fbd0a9 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:03.330670 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1180/2350] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:03 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:03.332234 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cee46-a0eb-47bf-99e8-41750a81194e None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:03.343570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cee46-a0eb-47bf-99e8-41750a81194e None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:03.369473 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:03.403639 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfb433-c3b8-4781-aa63-2ef491b69660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:03.404284 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfb433-c3b8-4781-aa63-2ef491b69660 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:03.404562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfb433-c3b8-4781-aa63-2ef491b69660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:03.404834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfb433-c3b8-4781-aa63-2ef491b69660 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:03.405463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cfb433-c3b8-4781-aa63-2ef491b69660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:03.407439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cfb433-c3b8-4781-aa63-2ef491b69660 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:03.432912 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1171/2351] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:03 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:03.437346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfbab0d-0a44-42d7-9a1e-8cf2b1fbd0a9 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:03.450879 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfbab0d-0a44-42d7-9a1e-8cf2b1fbd0a9 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:03.456266 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75dbf0-7698-471e-afd0-a4cfcae7bf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:03.456891 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75dbf0-7698-471e-afd0-a4cfcae7bf17 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:03.457175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75dbf0-7698-471e-afd0-a4cfcae7bf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:03.457451 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75dbf0-7698-471e-afd0-a4cfcae7bf17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:03.458101 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75dbf0-7698-471e-afd0-a4cfcae7bf17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:03.459801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75dbf0-7698-471e-afd0-a4cfcae7bf17 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:03.471010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1181/2352] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:03.475588 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfb433-c3b8-4781-aa63-2ef491b69660 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:03.489030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfb433-c3b8-4781-aa63-2ef491b69660 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:03.494254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2824e-8213-43a2-a60d-07eea7e5dbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:03.494866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2824e-8213-43a2-a60d-07eea7e5dbde admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:03.495145 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2824e-8213-43a2-a60d-07eea7e5dbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:03.495412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2824e-8213-43a2-a60d-07eea7e5dbde admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:03.496146 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2824e-8213-43a2-a60d-07eea7e5dbde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:03.497764 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2824e-8213-43a2-a60d-07eea7e5dbde admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:03.509215 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1172/2353] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:32:03.513680 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75dbf0-7698-471e-afd0-a4cfcae7bf17 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:03.524198 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75dbf0-7698-471e-afd0-a4cfcae7bf17 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:03.529150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86669b-1b06-4276-8692-f9b985ac65a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:03.529821 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86669b-1b06-4276-8692-f9b985ac65a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:03.530098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86669b-1b06-4276-8692-f9b985ac65a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:03.530403 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a86669b-1b06-4276-8692-f9b985ac65a0 admin admin] PATH_INFO: `/v3/projects/c79cc68a8e4c498580e580a4790a7ed5/users/c8f868d76e3f465ea7803009a8a9a7fa/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:03.546574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a86669b-1b06-4276-8692-f9b985ac65a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c79cc68a8e4c498580e580a4790a7ed5, user_id=c8f868d76e3f465ea7803009a8a9a7fa, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:03.549361 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a86669b-1b06-4276-8692-f9b985ac65a0 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:03.564260 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1182/2354] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:03 2022] PUT /identity/v3/projects/c79cc68a8e4c498580e580a4790a7ed5/users/c8f868d76e3f465ea7803009a8a9a7fa/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:03.574733 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016a19c-4fac-4755-9ead-785f10f02a8b None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:03.575023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016a19c-4fac-4755-9ead-785f10f02a8b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:03.575301 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016a19c-4fac-4755-9ead-785f10f02a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:03.603183 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-4016a19c-4fac-4755-9ead-785f10f02a8b None None] MFA Rules not processed for user `c8f868d76e3f465ea7803009a8a9a7fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:03.604447 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4016a19c-4fac-4755-9ead-785f10f02a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:03.676829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1173/2355] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:03.681696 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86669b-1b06-4276-8692-f9b985ac65a0 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:03.763379 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a86669b-1b06-4276-8692-f9b985ac65a0 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:03.773594 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ebaa9-fa94-420e-afc0-662eea22a94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:03.774470 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ebaa9-fa94-420e-afc0-662eea22a94e admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:03.774752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ebaa9-fa94-420e-afc0-662eea22a94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:03.775055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ebaa9-fa94-420e-afc0-662eea22a94e admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:03.775915 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ebaa9-fa94-420e-afc0-662eea22a94e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:03.778939 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ebaa9-fa94-420e-afc0-662eea22a94e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:03.810184 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1183/2356] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:03 2022] POST /identity/v3/projects => generated 377 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:03.814542 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016a19c-4fac-4755-9ead-785f10f02a8b None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:03.825565 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016a19c-4fac-4755-9ead-785f10f02a8b None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:03.895299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48338ede-960a-4d5e-92ba-1ba085d9eac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:03.896012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48338ede-960a-4d5e-92ba-1ba085d9eac5 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:03.896346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48338ede-960a-4d5e-92ba-1ba085d9eac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:03.896827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48338ede-960a-4d5e-92ba-1ba085d9eac5 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:03.897579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48338ede-960a-4d5e-92ba-1ba085d9eac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:03.899531 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48338ede-960a-4d5e-92ba-1ba085d9eac5 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:03.927478 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1174/2357] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:03 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:03.928608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ebaa9-fa94-420e-afc0-662eea22a94e admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:03.940218 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ebaa9-fa94-420e-afc0-662eea22a94e admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:03.945562 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c18dc3-0b9d-4a1b-8894-5d51ce34992d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:03.946355 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c18dc3-0b9d-4a1b-8894-5d51ce34992d admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:03.947077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c18dc3-0b9d-4a1b-8894-5d51ce34992d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:03.947077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c18dc3-0b9d-4a1b-8894-5d51ce34992d admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:03.947883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c18dc3-0b9d-4a1b-8894-5d51ce34992d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:03.949923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c18dc3-0b9d-4a1b-8894-5d51ce34992d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:03.963793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1184/2358] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:03.968075 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48338ede-960a-4d5e-92ba-1ba085d9eac5 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:03.979516 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48338ede-960a-4d5e-92ba-1ba085d9eac5 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:03.985845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad4926-30f6-43d2-9fe4-0322edbd9a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:03.986549 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad4926-30f6-43d2-9fe4-0322edbd9a1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:03.986896 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad4926-30f6-43d2-9fe4-0322edbd9a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:03.987233 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad4926-30f6-43d2-9fe4-0322edbd9a1b admin admin] PATH_INFO: `/v3/projects/0557615f96cf48e98959e29ac3799b99/users/029e17d242d24f8c83222e0be5d873a3/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:04.005403 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad4926-30f6-43d2-9fe4-0322edbd9a1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0557615f96cf48e98959e29ac3799b99, user_id=029e17d242d24f8c83222e0be5d873a3, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:04.008436 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad4926-30f6-43d2-9fe4-0322edbd9a1b admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:04.021502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:04.027253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1175/2359] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:03 2022] PUT /identity/v3/projects/0557615f96cf48e98959e29ac3799b99/users/029e17d242d24f8c83222e0be5d873a3/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:04.031373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c18dc3-0b9d-4a1b-8894-5d51ce34992d admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:04.042844 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c18dc3-0b9d-4a1b-8894-5d51ce34992d admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:04.104924 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c08c5e2-d897-4075-9206-f105064066dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:04.105546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08c5e2-d897-4075-9206-f105064066dc admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:04.105864 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08c5e2-d897-4075-9206-f105064066dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:04.106140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c08c5e2-d897-4075-9206-f105064066dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:04.106790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08c5e2-d897-4075-9206-f105064066dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:04.108429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c08c5e2-d897-4075-9206-f105064066dc admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:04.110596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:04.120152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1185/2360] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:04.124422 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad4926-30f6-43d2-9fe4-0322edbd9a1b admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:04.135517 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad4926-30f6-43d2-9fe4-0322edbd9a1b admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:04.140904 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975f9cc-d7de-4b7c-9007-d4d884af6c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:04.141690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975f9cc-d7de-4b7c-9007-d4d884af6c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:04.141970 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975f9cc-d7de-4b7c-9007-d4d884af6c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:04.142269 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975f9cc-d7de-4b7c-9007-d4d884af6c06 admin admin] PATH_INFO: `/v3/projects/0557615f96cf48e98959e29ac3799b99/users/029e17d242d24f8c83222e0be5d873a3/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:04.144371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975f9cc-d7de-4b7c-9007-d4d884af6c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0557615f96cf48e98959e29ac3799b99, user_id=029e17d242d24f8c83222e0be5d873a3, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:04.147023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975f9cc-d7de-4b7c-9007-d4d884af6c06 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:04.158474 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1176/2361] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:04 2022] PUT /identity/v3/projects/0557615f96cf48e98959e29ac3799b99/users/029e17d242d24f8c83222e0be5d873a3/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:04.175125 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda36fe-1bc2-4efb-829b-4d205fbaef5d None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:04.175409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda36fe-1bc2-4efb-829b-4d205fbaef5d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:04.175679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda36fe-1bc2-4efb-829b-4d205fbaef5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:04.205791 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-5eda36fe-1bc2-4efb-829b-4d205fbaef5d None None] MFA Rules not processed for user `029e17d242d24f8c83222e0be5d873a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:04.207029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5eda36fe-1bc2-4efb-829b-4d205fbaef5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:04.276536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1186/2362] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:04.293661 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975f9cc-d7de-4b7c-9007-d4d884af6c06 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:04.297003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e975f9cc-d7de-4b7c-9007-d4d884af6c06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:04.361947 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975f9cc-d7de-4b7c-9007-d4d884af6c06 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:04.366416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9d28d-fd74-4538-963d-f2c336347b5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:04.367034 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9d28d-fd74-4538-963d-f2c336347b5e service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:04.367342 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9d28d-fd74-4538-963d-f2c336347b5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:04.367610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9d28d-fd74-4538-963d-f2c336347b5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:04.378043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db9d28d-fd74-4538-963d-f2c336347b5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:04.379501 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db9d28d-fd74-4538-963d-f2c336347b5e service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:04.389756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1177/2363] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:04.436625 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda36fe-1bc2-4efb-829b-4d205fbaef5d None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:04.438230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5eda36fe-1bc2-4efb-829b-4d205fbaef5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:04.451536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda36fe-1bc2-4efb-829b-4d205fbaef5d None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:04.455125 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb5338-08a2-4787-b750-e884f9ad2b55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:04.456216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb5338-08a2-4787-b750-e884f9ad2b55 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:04.456216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb5338-08a2-4787-b750-e884f9ad2b55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:04.456438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb5338-08a2-4787-b750-e884f9ad2b55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:04.515694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb5338-08a2-4787-b750-e884f9ad2b55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:04.517232 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb5338-08a2-4787-b750-e884f9ad2b55 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:04.526735 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1187/2364] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:07.479349 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9d28d-fd74-4538-963d-f2c336347b5e service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:07.553266 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9d28d-fd74-4538-963d-f2c336347b5e service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:07.557978 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362dcb0-b683-4660-8e0e-2e7580fb96f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:07.558710 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362dcb0-b683-4660-8e0e-2e7580fb96f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:07.559079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362dcb0-b683-4660-8e0e-2e7580fb96f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:07.559447 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362dcb0-b683-4660-8e0e-2e7580fb96f1 admin admin] PATH_INFO: `/v3/users/bad9100b16964faea77f1ad9e5eca214` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:07.564980 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362dcb0-b683-4660-8e0e-2e7580fb96f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bad9100b16964faea77f1ad9e5eca214)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:07.566946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362dcb0-b683-4660-8e0e-2e7580fb96f1 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:07.621304 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0362dcb0-b683-4660-8e0e-2e7580fb96f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bad9100b16964faea77f1ad9e5eca214'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:07.628853 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0362dcb0-b683-4660-8e0e-2e7580fb96f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bad9100b16964faea77f1ad9e5eca214'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:07.641229 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0362dcb0-b683-4660-8e0e-2e7580fb96f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bad9100b16964faea77f1ad9e5eca214'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:07.641946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0362dcb0-b683-4660-8e0e-2e7580fb96f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bad9100b16964faea77f1ad9e5eca214'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:07.650242 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0362dcb0-b683-4660-8e0e-2e7580fb96f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bad9100b16964faea77f1ad9e5eca214'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:07.656782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1178/2365] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:07 2022] DELETE /identity/v3/users/bad9100b16964faea77f1ad9e5eca214 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:07.661386 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e8dd2-d5d6-4d45-9a1d-9d0f3087199c None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:07.661880 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e8dd2-d5d6-4d45-9a1d-9d0f3087199c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:07.661976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e8dd2-d5d6-4d45-9a1d-9d0f3087199c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:07.681609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-5d5e8dd2-d5d6-4d45-9a1d-9d0f3087199c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:07.682810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5d5e8dd2-d5d6-4d45-9a1d-9d0f3087199c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:07.742709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1188/2366] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:07.752805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362dcb0-b683-4660-8e0e-2e7580fb96f1 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:07.754368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0362dcb0-b683-4660-8e0e-2e7580fb96f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:07.766279 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:07.823876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362dcb0-b683-4660-8e0e-2e7580fb96f1 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:07.828472 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abdd723-616f-4828-965f-97cc6ca2252e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:07.828932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abdd723-616f-4828-965f-97cc6ca2252e service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:07.829220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abdd723-616f-4828-965f-97cc6ca2252e service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:07.829467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abdd723-616f-4828-965f-97cc6ca2252e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:07.840545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abdd723-616f-4828-965f-97cc6ca2252e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:07.842400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abdd723-616f-4828-965f-97cc6ca2252e service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:07.853359 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1179/2367] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:08.141601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e8dd2-d5d6-4d45-9a1d-9d0f3087199c None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:08.143271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5d5e8dd2-d5d6-4d45-9a1d-9d0f3087199c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:08.156244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e8dd2-d5d6-4d45-9a1d-9d0f3087199c None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:08.161708 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3023b-51b3-45b3-b118-3883b151087e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:08.162372 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3023b-51b3-45b3-b118-3883b151087e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:08.162796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3023b-51b3-45b3-b118-3883b151087e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:08.163078 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3023b-51b3-45b3-b118-3883b151087e admin admin] PATH_INFO: `/v3/projects/a534f5b4d1d743cba1b7222fa173c73d` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:08.164276 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3023b-51b3-45b3-b118-3883b151087e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a534f5b4d1d743cba1b7222fa173c73d)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:08.166305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3023b-51b3-45b3-b118-3883b151087e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:08.184990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:08.217135 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-15c3023b-51b3-45b3-b118-3883b151087e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a534f5b4d1d743cba1b7222fa173c73d', 'request_id': 'req-15c3023b-51b3-45b3-b118-3883b151087e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:08.223482 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-15c3023b-51b3-45b3-b118-3883b151087e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a534f5b4d1d743cba1b7222fa173c73d', 'request_id': 'req-15c3023b-51b3-45b3-b118-3883b151087e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:08.233604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-15c3023b-51b3-45b3-b118-3883b151087e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a534f5b4d1d743cba1b7222fa173c73d', 'request_id': 'req-15c3023b-51b3-45b3-b118-3883b151087e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:08.246702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-15c3023b-51b3-45b3-b118-3883b151087e admin admin] The token cache is being invalidate because project a534f5b4d1d743cba1b7222fa173c73d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:08.247305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-15c3023b-51b3-45b3-b118-3883b151087e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:08.248396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-15c3023b-51b3-45b3-b118-3883b151087e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:08.250232 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1189/2368] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:08 2022] DELETE /identity/v3/projects/a534f5b4d1d743cba1b7222fa173c73d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:08.254379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abdd723-616f-4828-965f-97cc6ca2252e service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:08.264725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abdd723-616f-4828-965f-97cc6ca2252e service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:08.325134 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f85fe-dab3-40f8-9d7e-6165431f0e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:08.325774 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f85fe-dab3-40f8-9d7e-6165431f0e45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:08.326064 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f85fe-dab3-40f8-9d7e-6165431f0e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:08.326355 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f85fe-dab3-40f8-9d7e-6165431f0e45 admin admin] PATH_INFO: `/v3/users/c8f868d76e3f465ea7803009a8a9a7fa` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:08.327496 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f85fe-dab3-40f8-9d7e-6165431f0e45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8f868d76e3f465ea7803009a8a9a7fa)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:08.329533 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f85fe-dab3-40f8-9d7e-6165431f0e45 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:08.376811 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-333f85fe-dab3-40f8-9d7e-6165431f0e45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8f868d76e3f465ea7803009a8a9a7fa'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:08.383637 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-333f85fe-dab3-40f8-9d7e-6165431f0e45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8f868d76e3f465ea7803009a8a9a7fa'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:08.396045 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-333f85fe-dab3-40f8-9d7e-6165431f0e45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8f868d76e3f465ea7803009a8a9a7fa'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:08.396778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-333f85fe-dab3-40f8-9d7e-6165431f0e45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8f868d76e3f465ea7803009a8a9a7fa'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:08.405211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-333f85fe-dab3-40f8-9d7e-6165431f0e45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8f868d76e3f465ea7803009a8a9a7fa'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:08.412381 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1180/2369] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:08 2022] DELETE /identity/v3/users/c8f868d76e3f465ea7803009a8a9a7fa => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:08.739227 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3023b-51b3-45b3-b118-3883b151087e admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:08.741848 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-15c3023b-51b3-45b3-b118-3883b151087e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:08.766560 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3023b-51b3-45b3-b118-3883b151087e admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:08.833841 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42225044-6ade-4be8-b4fd-200685a660d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:08.834484 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42225044-6ade-4be8-b4fd-200685a660d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:08.834760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42225044-6ade-4be8-b4fd-200685a660d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:08.835030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42225044-6ade-4be8-b4fd-200685a660d0 admin admin] PATH_INFO: `/v3/projects/c79cc68a8e4c498580e580a4790a7ed5` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:08.836175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42225044-6ade-4be8-b4fd-200685a660d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c79cc68a8e4c498580e580a4790a7ed5)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:08.838168 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42225044-6ade-4be8-b4fd-200685a660d0 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:08.890374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-42225044-6ade-4be8-b4fd-200685a660d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c79cc68a8e4c498580e580a4790a7ed5', 'request_id': 'req-42225044-6ade-4be8-b4fd-200685a660d0'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:08.899697 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-42225044-6ade-4be8-b4fd-200685a660d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c79cc68a8e4c498580e580a4790a7ed5', 'request_id': 'req-42225044-6ade-4be8-b4fd-200685a660d0'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:08.908707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-42225044-6ade-4be8-b4fd-200685a660d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c79cc68a8e4c498580e580a4790a7ed5', 'request_id': 'req-42225044-6ade-4be8-b4fd-200685a660d0'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:08.919786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-42225044-6ade-4be8-b4fd-200685a660d0 admin admin] The token cache is being invalidate because project c79cc68a8e4c498580e580a4790a7ed5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:08.920493 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-42225044-6ade-4be8-b4fd-200685a660d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:08.921228 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-42225044-6ade-4be8-b4fd-200685a660d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:08.922761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1190/2370] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:08 2022] DELETE /identity/v3/projects/c79cc68a8e4c498580e580a4790a7ed5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:08.926933 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f85fe-dab3-40f8-9d7e-6165431f0e45 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:08.928491 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-333f85fe-dab3-40f8-9d7e-6165431f0e45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:08.941860 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f85fe-dab3-40f8-9d7e-6165431f0e45 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:08.972114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:09.002312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dea09c-ed3d-4403-bd53-5a503d949959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:09.003136 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dea09c-ed3d-4403-bd53-5a503d949959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:09.003355 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dea09c-ed3d-4403-bd53-5a503d949959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:09.003628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dea09c-ed3d-4403-bd53-5a503d949959 admin admin] PATH_INFO: `/v3/users/029e17d242d24f8c83222e0be5d873a3` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:09.004824 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dea09c-ed3d-4403-bd53-5a503d949959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=029e17d242d24f8c83222e0be5d873a3)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:09.006775 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dea09c-ed3d-4403-bd53-5a503d949959 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:09.058921 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-00dea09c-ed3d-4403-bd53-5a503d949959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '029e17d242d24f8c83222e0be5d873a3'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:09.067038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-00dea09c-ed3d-4403-bd53-5a503d949959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '029e17d242d24f8c83222e0be5d873a3'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:09.079818 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-00dea09c-ed3d-4403-bd53-5a503d949959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '029e17d242d24f8c83222e0be5d873a3'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:09.080589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-00dea09c-ed3d-4403-bd53-5a503d949959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '029e17d242d24f8c83222e0be5d873a3'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:09.093151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-00dea09c-ed3d-4403-bd53-5a503d949959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '029e17d242d24f8c83222e0be5d873a3'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:09.100390 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1181/2371] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:08 2022] DELETE /identity/v3/users/029e17d242d24f8c83222e0be5d873a3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:09.389413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42225044-6ade-4be8-b4fd-200685a660d0 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:09.390951 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-42225044-6ade-4be8-b4fd-200685a660d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:09.405314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42225044-6ade-4be8-b4fd-200685a660d0 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:09.444196 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:09.472699 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ab7be-f381-43b8-ab91-3a046f43ff8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:09.473024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ab7be-f381-43b8-ab91-3a046f43ff8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:09.473216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ab7be-f381-43b8-ab91-3a046f43ff8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:09.473489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ab7be-f381-43b8-ab91-3a046f43ff8e admin admin] PATH_INFO: `/v3/projects/0557615f96cf48e98959e29ac3799b99` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:09.474664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ab7be-f381-43b8-ab91-3a046f43ff8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0557615f96cf48e98959e29ac3799b99)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:09.476651 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ab7be-f381-43b8-ab91-3a046f43ff8e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:09.531288 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1b2ab7be-f381-43b8-ab91-3a046f43ff8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0557615f96cf48e98959e29ac3799b99', 'request_id': 'req-1b2ab7be-f381-43b8-ab91-3a046f43ff8e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:09.540477 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1b2ab7be-f381-43b8-ab91-3a046f43ff8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0557615f96cf48e98959e29ac3799b99', 'request_id': 'req-1b2ab7be-f381-43b8-ab91-3a046f43ff8e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:09.549632 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1b2ab7be-f381-43b8-ab91-3a046f43ff8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0557615f96cf48e98959e29ac3799b99', 'request_id': 'req-1b2ab7be-f381-43b8-ab91-3a046f43ff8e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:09.560720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1b2ab7be-f381-43b8-ab91-3a046f43ff8e admin admin] The token cache is being invalidate because project 0557615f96cf48e98959e29ac3799b99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:09.561009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1b2ab7be-f381-43b8-ab91-3a046f43ff8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:09.561639 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1b2ab7be-f381-43b8-ab91-3a046f43ff8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:09.563152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1191/2372] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:09 2022] DELETE /identity/v3/projects/0557615f96cf48e98959e29ac3799b99 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:09.569466 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32d52c-c35e-4ed5-b87b-a1bbb0a3f35f None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:09.569770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32d52c-c35e-4ed5-b87b-a1bbb0a3f35f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:09.570089 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32d52c-c35e-4ed5-b87b-a1bbb0a3f35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:09.590198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-ff32d52c-c35e-4ed5-b87b-a1bbb0a3f35f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:09.591570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-ff32d52c-c35e-4ed5-b87b-a1bbb0a3f35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:09.654017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1182/2373] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:09.663006 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40e9c6-c1c8-49be-b7c5-eab5204c0c49 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:09.663367 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40e9c6-c1c8-49be-b7c5-eab5204c0c49 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:09.663711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc40e9c6-c1c8-49be-b7c5-eab5204c0c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:09.684878 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-dc40e9c6-c1c8-49be-b7c5-eab5204c0c49 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:09.686659 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-dc40e9c6-c1c8-49be-b7c5-eab5204c0c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:09.695103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1192/2374] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:09.699671 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32d52c-c35e-4ed5-b87b-a1bbb0a3f35f None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:09.709511 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32d52c-c35e-4ed5-b87b-a1bbb0a3f35f None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:09.714161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e133b7-7913-4991-8caa-b5979016f617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:09.714771 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e133b7-7913-4991-8caa-b5979016f617 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:09.715046 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e133b7-7913-4991-8caa-b5979016f617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:09.715309 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e133b7-7913-4991-8caa-b5979016f617 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:09.715950 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e133b7-7913-4991-8caa-b5979016f617 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:32:09.716277 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e133b7-7913-4991-8caa-b5979016f617 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:09.717881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e133b7-7913-4991-8caa-b5979016f617 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:09.732460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1183/2375] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:32:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:32:09.737410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40e9c6-c1c8-49be-b7c5-eab5204c0c49 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:09.748448 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc40e9c6-c1c8-49be-b7c5-eab5204c0c49 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:09.754317 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a493e-5b6d-44ae-9267-8601a7d7fa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:09.754936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a493e-5b6d-44ae-9267-8601a7d7fa69 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:09.755202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a493e-5b6d-44ae-9267-8601a7d7fa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:09.755473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a493e-5b6d-44ae-9267-8601a7d7fa69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:09.756211 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a493e-5b6d-44ae-9267-8601a7d7fa69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:09.758059 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a493e-5b6d-44ae-9267-8601a7d7fa69 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:09.793075 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1193/2376] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:09 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:09.797856 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e133b7-7913-4991-8caa-b5979016f617 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:09.811316 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e133b7-7913-4991-8caa-b5979016f617 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:09.850442 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:09.877693 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e9dcc-2193-445a-914e-0c78956b836a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:09.878321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e9dcc-2193-445a-914e-0c78956b836a admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:09.878598 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e9dcc-2193-445a-914e-0c78956b836a admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:09.878868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247e9dcc-2193-445a-914e-0c78956b836a admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:09.879498 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e9dcc-2193-445a-914e-0c78956b836a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:09.881457 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247e9dcc-2193-445a-914e-0c78956b836a admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:09.910177 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1184/2377] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:09 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:09.911668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a493e-5b6d-44ae-9267-8601a7d7fa69 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:09.922379 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a493e-5b6d-44ae-9267-8601a7d7fa69 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:09.927121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ef24e-69ee-4a90-a499-6993fb89557a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:09.927728 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ef24e-69ee-4a90-a499-6993fb89557a admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:09.928032 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ef24e-69ee-4a90-a499-6993fb89557a admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:09.928309 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ef24e-69ee-4a90-a499-6993fb89557a admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:09.928953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ef24e-69ee-4a90-a499-6993fb89557a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:09.930574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ef24e-69ee-4a90-a499-6993fb89557a admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:09.941182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1194/2378] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:09.945453 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e9dcc-2193-445a-914e-0c78956b836a admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:09.955973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247e9dcc-2193-445a-914e-0c78956b836a admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:09.964646 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46fb50-7009-4d1f-bd88-08411d0e25ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:09.965425 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46fb50-7009-4d1f-bd88-08411d0e25ee admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:09.965832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46fb50-7009-4d1f-bd88-08411d0e25ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:09.966203 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46fb50-7009-4d1f-bd88-08411d0e25ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:09.967063 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46fb50-7009-4d1f-bd88-08411d0e25ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:09.969017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46fb50-7009-4d1f-bd88-08411d0e25ee admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:09.980386 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1185/2379] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:09.987735 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ef24e-69ee-4a90-a499-6993fb89557a admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:09.999645 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ef24e-69ee-4a90-a499-6993fb89557a admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:10.004134 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a70d0-58fa-4a4a-ab91-71d688217edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:10.004990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a70d0-58fa-4a4a-ab91-71d688217edd admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:10.005273 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a70d0-58fa-4a4a-ab91-71d688217edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:10.005549 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a70d0-58fa-4a4a-ab91-71d688217edd admin admin] PATH_INFO: `/v3/projects/717edee5365d4d928c9cf4be098199ea/users/22804d971abc4b48aa3416106af83192/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:10.022865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a70d0-58fa-4a4a-ab91-71d688217edd admin admin] RBAC: Authorizing `identity:create_grant(project_id=717edee5365d4d928c9cf4be098199ea, user_id=22804d971abc4b48aa3416106af83192, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:10.025578 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a70d0-58fa-4a4a-ab91-71d688217edd admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:10.042149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1195/2380] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:09 2022] PUT /identity/v3/projects/717edee5365d4d928c9cf4be098199ea/users/22804d971abc4b48aa3416106af83192/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:10.054162 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c6d5b-06df-4712-b29d-9cd3314d3067 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:10.054456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c6d5b-06df-4712-b29d-9cd3314d3067 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:10.054786 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c6d5b-06df-4712-b29d-9cd3314d3067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:10.083782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-777c6d5b-06df-4712-b29d-9cd3314d3067 None None] MFA Rules not processed for user `22804d971abc4b48aa3416106af83192`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:10.086293 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-777c6d5b-06df-4712-b29d-9cd3314d3067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:10.155928 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1186/2381] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:10.160613 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a70d0-58fa-4a4a-ab91-71d688217edd admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:10.225576 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a70d0-58fa-4a4a-ab91-71d688217edd admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:10.231416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f8b53-4b21-4c52-893c-61410b3a6041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:10.232111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f8b53-4b21-4c52-893c-61410b3a6041 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:10.232396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f8b53-4b21-4c52-893c-61410b3a6041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:10.232673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f8b53-4b21-4c52-893c-61410b3a6041 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:10.233316 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f8b53-4b21-4c52-893c-61410b3a6041 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:10.235316 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f8b53-4b21-4c52-893c-61410b3a6041 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:10.245390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:10.267097 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1196/2382] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:10 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:10.271491 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c6d5b-06df-4712-b29d-9cd3314d3067 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:10.283707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c6d5b-06df-4712-b29d-9cd3314d3067 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:10.342000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04d84c-de8b-4667-8c35-7b14b0c57032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:10.342656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04d84c-de8b-4667-8c35-7b14b0c57032 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:10.342935 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04d84c-de8b-4667-8c35-7b14b0c57032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:10.343202 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04d84c-de8b-4667-8c35-7b14b0c57032 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:10.343869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04d84c-de8b-4667-8c35-7b14b0c57032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:10.345902 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04d84c-de8b-4667-8c35-7b14b0c57032 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:10.372523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1187/2383] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:10 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:10.373913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f8b53-4b21-4c52-893c-61410b3a6041 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:10.385803 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f8b53-4b21-4c52-893c-61410b3a6041 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:10.391066 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68e331-da43-4880-82b3-a68b81451a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:10.391707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68e331-da43-4880-82b3-a68b81451a2c admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:10.392018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68e331-da43-4880-82b3-a68b81451a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:10.392348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68e331-da43-4880-82b3-a68b81451a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:10.393021 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68e331-da43-4880-82b3-a68b81451a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:10.394685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f68e331-da43-4880-82b3-a68b81451a2c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:10.406283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1197/2384] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:10.409640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04d84c-de8b-4667-8c35-7b14b0c57032 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:10.420885 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04d84c-de8b-4667-8c35-7b14b0c57032 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:10.427606 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfb393-58e7-492c-b266-5d8fda04c7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:10.428322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfb393-58e7-492c-b266-5d8fda04c7c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:10.428591 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfb393-58e7-492c-b266-5d8fda04c7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:10.428854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbfb393-58e7-492c-b266-5d8fda04c7c7 admin admin] PATH_INFO: `/v3/projects/50e9d62fbbbf4d1d80265511de49b00e/users/374fb31b9fe04a00883c6b063268a8a0/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:10.448190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfb393-58e7-492c-b266-5d8fda04c7c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50e9d62fbbbf4d1d80265511de49b00e, user_id=374fb31b9fe04a00883c6b063268a8a0, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:10.450866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbfb393-58e7-492c-b266-5d8fda04c7c7 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:10.466851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1188/2385] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:10 2022] PUT /identity/v3/projects/50e9d62fbbbf4d1d80265511de49b00e/users/374fb31b9fe04a00883c6b063268a8a0/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:10.471011 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68e331-da43-4880-82b3-a68b81451a2c admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:10.481471 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f68e331-da43-4880-82b3-a68b81451a2c admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:10.541581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbb75e9-3eb3-4c26-a2d3-981e938e1f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:10.542244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb75e9-3eb3-4c26-a2d3-981e938e1f3e admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:10.542677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb75e9-3eb3-4c26-a2d3-981e938e1f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:10.543057 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbb75e9-3eb3-4c26-a2d3-981e938e1f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:10.543716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb75e9-3eb3-4c26-a2d3-981e938e1f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:10.545775 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbb75e9-3eb3-4c26-a2d3-981e938e1f3e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:10.557212 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1198/2386] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:10.561638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfb393-58e7-492c-b266-5d8fda04c7c7 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:10.571923 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbfb393-58e7-492c-b266-5d8fda04c7c7 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:10.576568 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fc7d8-c4b3-448b-bb15-4638d864aaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:10.577240 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fc7d8-c4b3-448b-bb15-4638d864aaed admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:10.577527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fc7d8-c4b3-448b-bb15-4638d864aaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:10.577800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fc7d8-c4b3-448b-bb15-4638d864aaed admin admin] PATH_INFO: `/v3/projects/50e9d62fbbbf4d1d80265511de49b00e/users/374fb31b9fe04a00883c6b063268a8a0/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:10.579715 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fc7d8-c4b3-448b-bb15-4638d864aaed admin admin] RBAC: Authorizing `identity:create_grant(project_id=50e9d62fbbbf4d1d80265511de49b00e, user_id=374fb31b9fe04a00883c6b063268a8a0, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:10.582491 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fc7d8-c4b3-448b-bb15-4638d864aaed admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:10.588507 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:10.595975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1189/2387] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:10 2022] PUT /identity/v3/projects/50e9d62fbbbf4d1d80265511de49b00e/users/374fb31b9fe04a00883c6b063268a8a0/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:10.607313 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ab123-e20a-4425-996f-3cee73f07fc9 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:10.607676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ab123-e20a-4425-996f-3cee73f07fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:10.607976 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ab123-e20a-4425-996f-3cee73f07fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:10.637747 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-799ab123-e20a-4425-996f-3cee73f07fc9 None None] MFA Rules not processed for user `374fb31b9fe04a00883c6b063268a8a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:10.638961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-799ab123-e20a-4425-996f-3cee73f07fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:10.711404 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1199/2388] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:10.723039 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fc7d8-c4b3-448b-bb15-4638d864aaed admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:10.725203 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c74fc7d8-c4b3-448b-bb15-4638d864aaed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:10.916801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fc7d8-c4b3-448b-bb15-4638d864aaed admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:10.920821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285aa97d-67bf-4f96-b20a-4af97a914343 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:10.921430 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285aa97d-67bf-4f96-b20a-4af97a914343 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:10.921708 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285aa97d-67bf-4f96-b20a-4af97a914343 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:10.921981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285aa97d-67bf-4f96-b20a-4af97a914343 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:10.931543 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285aa97d-67bf-4f96-b20a-4af97a914343 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:10.933016 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285aa97d-67bf-4f96-b20a-4af97a914343 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:10.948428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1190/2389] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:11.606914 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ab123-e20a-4425-996f-3cee73f07fc9 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:11.668039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:11.673376 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ab123-e20a-4425-996f-3cee73f07fc9 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:11.679725 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf6499-087b-46c9-973b-c8a8f31725b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:11.680572 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf6499-087b-46c9-973b-c8a8f31725b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:11.680928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf6499-087b-46c9-973b-c8a8f31725b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:11.681389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf6499-087b-46c9-973b-c8a8f31725b5 admin admin] PATH_INFO: `/v3/users/22804d971abc4b48aa3416106af83192` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:11.682643 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf6499-087b-46c9-973b-c8a8f31725b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22804d971abc4b48aa3416106af83192)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:11.684674 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf6499-087b-46c9-973b-c8a8f31725b5 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:11.733162 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-baaf6499-087b-46c9-973b-c8a8f31725b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22804d971abc4b48aa3416106af83192'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:11.745036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-baaf6499-087b-46c9-973b-c8a8f31725b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22804d971abc4b48aa3416106af83192'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:11.751626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-baaf6499-087b-46c9-973b-c8a8f31725b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22804d971abc4b48aa3416106af83192'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:11.752362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-baaf6499-087b-46c9-973b-c8a8f31725b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22804d971abc4b48aa3416106af83192'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:11.752962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-baaf6499-087b-46c9-973b-c8a8f31725b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22804d971abc4b48aa3416106af83192'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:11.768837 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1200/2390] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:11 2022] DELETE /identity/v3/users/22804d971abc4b48aa3416106af83192 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:11.773509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd336f8-82f3-4e19-9bee-ed1b9d5682f1 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:11.773800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd336f8-82f3-4e19-9bee-ed1b9d5682f1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:11.774052 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd336f8-82f3-4e19-9bee-ed1b9d5682f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:11.792891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-bdd336f8-82f3-4e19-9bee-ed1b9d5682f1 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:11.794099 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-bdd336f8-82f3-4e19-9bee-ed1b9d5682f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:11.869959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1191/2391] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:11.880894 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf6499-087b-46c9-973b-c8a8f31725b5 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:11.882485 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-baaf6499-087b-46c9-973b-c8a8f31725b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:11.950222 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf6499-087b-46c9-973b-c8a8f31725b5 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:11.954483 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2a678-a95f-4688-9b3c-4ba2478ae483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:11.955099 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2a678-a95f-4688-9b3c-4ba2478ae483 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:11.955380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2a678-a95f-4688-9b3c-4ba2478ae483 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:11.955655 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2a678-a95f-4688-9b3c-4ba2478ae483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:11.965711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2a678-a95f-4688-9b3c-4ba2478ae483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:11.967162 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2a678-a95f-4688-9b3c-4ba2478ae483 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:11.977534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1201/2392] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:12.262058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd336f8-82f3-4e19-9bee-ed1b9d5682f1 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:12.263566 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-bdd336f8-82f3-4e19-9bee-ed1b9d5682f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:12.277217 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd336f8-82f3-4e19-9bee-ed1b9d5682f1 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:12.282239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96739a11-04cc-4583-92cd-28a9c808be78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:12.282925 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96739a11-04cc-4583-92cd-28a9c808be78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:12.283203 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96739a11-04cc-4583-92cd-28a9c808be78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:12.283478 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96739a11-04cc-4583-92cd-28a9c808be78 admin admin] PATH_INFO: `/v3/projects/717edee5365d4d928c9cf4be098199ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:12.284742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96739a11-04cc-4583-92cd-28a9c808be78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=717edee5365d4d928c9cf4be098199ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:12.286788 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96739a11-04cc-4583-92cd-28a9c808be78 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:12.349178 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-96739a11-04cc-4583-92cd-28a9c808be78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '717edee5365d4d928c9cf4be098199ea', 'request_id': 'req-96739a11-04cc-4583-92cd-28a9c808be78'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:12.354770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-96739a11-04cc-4583-92cd-28a9c808be78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '717edee5365d4d928c9cf4be098199ea', 'request_id': 'req-96739a11-04cc-4583-92cd-28a9c808be78'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:12.363901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-96739a11-04cc-4583-92cd-28a9c808be78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '717edee5365d4d928c9cf4be098199ea', 'request_id': 'req-96739a11-04cc-4583-92cd-28a9c808be78'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:12.379359 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-96739a11-04cc-4583-92cd-28a9c808be78 admin admin] The token cache is being invalidate because project 717edee5365d4d928c9cf4be098199ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:12.379641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-96739a11-04cc-4583-92cd-28a9c808be78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:12.380340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-96739a11-04cc-4583-92cd-28a9c808be78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:12.381850 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1192/2393] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:12 2022] DELETE /identity/v3/projects/717edee5365d4d928c9cf4be098199ea => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:12.386125 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2a678-a95f-4688-9b3c-4ba2478ae483 service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:12.397183 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2a678-a95f-4688-9b3c-4ba2478ae483 service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:12.460102 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b36adf-13f9-432b-8260-dba130f1b29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:12.460647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b36adf-13f9-432b-8260-dba130f1b29a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:12.460926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b36adf-13f9-432b-8260-dba130f1b29a admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:12.461198 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b36adf-13f9-432b-8260-dba130f1b29a admin admin] PATH_INFO: `/v3/users/374fb31b9fe04a00883c6b063268a8a0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:12.462346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b36adf-13f9-432b-8260-dba130f1b29a admin admin] RBAC: Authorizing `identity:delete_user(user_id=374fb31b9fe04a00883c6b063268a8a0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:12.464313 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b36adf-13f9-432b-8260-dba130f1b29a admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:12.621900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:12.649958 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-c1b36adf-13f9-432b-8260-dba130f1b29a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '374fb31b9fe04a00883c6b063268a8a0'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:12.661587 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-c1b36adf-13f9-432b-8260-dba130f1b29a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '374fb31b9fe04a00883c6b063268a8a0'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:12.668244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-c1b36adf-13f9-432b-8260-dba130f1b29a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '374fb31b9fe04a00883c6b063268a8a0'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:12.668934 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-c1b36adf-13f9-432b-8260-dba130f1b29a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '374fb31b9fe04a00883c6b063268a8a0'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:12.669539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-c1b36adf-13f9-432b-8260-dba130f1b29a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '374fb31b9fe04a00883c6b063268a8a0'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:12.683838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1202/2394] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:12 2022] DELETE /identity/v3/users/374fb31b9fe04a00883c6b063268a8a0 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:32:12.972156 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96739a11-04cc-4583-92cd-28a9c808be78 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:12.973639 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-96739a11-04cc-4583-92cd-28a9c808be78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:12.986634 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96739a11-04cc-4583-92cd-28a9c808be78 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:13.007755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:13.046129 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e42ed2-345d-458f-8662-11c5ef0498b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:13.046756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e42ed2-345d-458f-8662-11c5ef0498b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:13.047028 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e42ed2-345d-458f-8662-11c5ef0498b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:13.047293 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e42ed2-345d-458f-8662-11c5ef0498b4 admin admin] PATH_INFO: `/v3/projects/50e9d62fbbbf4d1d80265511de49b00e` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:13.048420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e42ed2-345d-458f-8662-11c5ef0498b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50e9d62fbbbf4d1d80265511de49b00e)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:13.050322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e42ed2-345d-458f-8662-11c5ef0498b4 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:13.100330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-92e42ed2-345d-458f-8662-11c5ef0498b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50e9d62fbbbf4d1d80265511de49b00e', 'request_id': 'req-92e42ed2-345d-458f-8662-11c5ef0498b4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:13.105810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-92e42ed2-345d-458f-8662-11c5ef0498b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50e9d62fbbbf4d1d80265511de49b00e', 'request_id': 'req-92e42ed2-345d-458f-8662-11c5ef0498b4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:13.111939 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-92e42ed2-345d-458f-8662-11c5ef0498b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50e9d62fbbbf4d1d80265511de49b00e', 'request_id': 'req-92e42ed2-345d-458f-8662-11c5ef0498b4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:13.127002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-92e42ed2-345d-458f-8662-11c5ef0498b4 admin admin] The token cache is being invalidate because project 50e9d62fbbbf4d1d80265511de49b00e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:13.127291 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-92e42ed2-345d-458f-8662-11c5ef0498b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:13.127901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-92e42ed2-345d-458f-8662-11c5ef0498b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:13.129427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1193/2395] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:12 2022] DELETE /identity/v3/projects/50e9d62fbbbf4d1d80265511de49b00e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:13.134585 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6378b0-f9e2-4088-bd57-c349526a1aac None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:13.134871 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6378b0-f9e2-4088-bd57-c349526a1aac None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:13.135147 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6378b0-f9e2-4088-bd57-c349526a1aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:13.155492 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-8c6378b0-f9e2-4088-bd57-c349526a1aac None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:13.156780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8c6378b0-f9e2-4088-bd57-c349526a1aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:13.220049 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1203/2396] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:13.229095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb694b-27c2-4c61-93c4-3292b5fea126 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:13.229385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb694b-27c2-4c61-93c4-3292b5fea126 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:13.229732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb694b-27c2-4c61-93c4-3292b5fea126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:13.251487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-d4eb694b-27c2-4c61-93c4-3292b5fea126 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:13.252738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-d4eb694b-27c2-4c61-93c4-3292b5fea126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:13.260655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1194/2397] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:13.265155 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6378b0-f9e2-4088-bd57-c349526a1aac None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:13.275295 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6378b0-f9e2-4088-bd57-c349526a1aac None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:13.280295 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2551420-2cb4-4b17-af90-f8e08c304798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:13.280904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2551420-2cb4-4b17-af90-f8e08c304798 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:13.281178 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2551420-2cb4-4b17-af90-f8e08c304798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:13.281443 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2551420-2cb4-4b17-af90-f8e08c304798 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:13.282084 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2551420-2cb4-4b17-af90-f8e08c304798 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:32:13.282384 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2551420-2cb4-4b17-af90-f8e08c304798 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:13.283962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2551420-2cb4-4b17-af90-f8e08c304798 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:13.300424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1204/2398] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:32:13.306583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb694b-27c2-4c61-93c4-3292b5fea126 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:13.316048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb694b-27c2-4c61-93c4-3292b5fea126 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:13.321287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f708f-d357-4900-b55a-bfcd499cfb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:13.321869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f708f-d357-4900-b55a-bfcd499cfb3f admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:13.322158 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f708f-d357-4900-b55a-bfcd499cfb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:13.322429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f708f-d357-4900-b55a-bfcd499cfb3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:13.323058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f708f-d357-4900-b55a-bfcd499cfb3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:13.325147 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f708f-d357-4900-b55a-bfcd499cfb3f admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:13.354666 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1195/2399] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:13 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:13.360134 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2551420-2cb4-4b17-af90-f8e08c304798 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:13.372413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2551420-2cb4-4b17-af90-f8e08c304798 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:13.437162 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81797f-c4bc-4144-861e-7ebd6eda1a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:13.437788 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81797f-c4bc-4144-861e-7ebd6eda1a68 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:13.438076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81797f-c4bc-4144-861e-7ebd6eda1a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:13.438341 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81797f-c4bc-4144-861e-7ebd6eda1a68 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:13.438986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81797f-c4bc-4144-861e-7ebd6eda1a68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:13.440947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81797f-c4bc-4144-861e-7ebd6eda1a68 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:13.463849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1205/2400] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:13 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:13.468136 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f708f-d357-4900-b55a-bfcd499cfb3f admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:13.478779 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f708f-d357-4900-b55a-bfcd499cfb3f admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:13.484041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c12e2-1385-43f3-830f-296e9b0baacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:13.484725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c12e2-1385-43f3-830f-296e9b0baacd admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:13.485026 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c12e2-1385-43f3-830f-296e9b0baacd admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:13.485272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c12e2-1385-43f3-830f-296e9b0baacd admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:13.485927 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c12e2-1385-43f3-830f-296e9b0baacd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:13.487554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c12e2-1385-43f3-830f-296e9b0baacd admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:13.499530 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1196/2401] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:13.503962 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81797f-c4bc-4144-861e-7ebd6eda1a68 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:13.514525 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81797f-c4bc-4144-861e-7ebd6eda1a68 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:13.519644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e16d6e-50c7-4243-a959-19694da74fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:13.520274 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e16d6e-50c7-4243-a959-19694da74fcb admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:13.520550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e16d6e-50c7-4243-a959-19694da74fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:13.520834 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e16d6e-50c7-4243-a959-19694da74fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:13.521461 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e16d6e-50c7-4243-a959-19694da74fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:13.523093 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e16d6e-50c7-4243-a959-19694da74fcb admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:13.534685 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1206/2402] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:13.539047 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c12e2-1385-43f3-830f-296e9b0baacd admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:13.549953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c12e2-1385-43f3-830f-296e9b0baacd admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:13.555982 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d1114-254f-427e-abaa-9896c9f3b197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:13.556710 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d1114-254f-427e-abaa-9896c9f3b197 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:13.556984 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d1114-254f-427e-abaa-9896c9f3b197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:13.557254 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d1114-254f-427e-abaa-9896c9f3b197 admin admin] PATH_INFO: `/v3/projects/869cb17c79de4ebdbec6086f0e52f1e3/users/01ec3acf20cc489da9dda278babb4e33/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:13.574372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d1114-254f-427e-abaa-9896c9f3b197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=869cb17c79de4ebdbec6086f0e52f1e3, user_id=01ec3acf20cc489da9dda278babb4e33, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:13.577232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d1114-254f-427e-abaa-9896c9f3b197 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:13.591429 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1197/2403] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:13 2022] PUT /identity/v3/projects/869cb17c79de4ebdbec6086f0e52f1e3/users/01ec3acf20cc489da9dda278babb4e33/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:13.603364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea0420-9636-40ec-8f37-e2a4dccf3590 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:13.603829 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea0420-9636-40ec-8f37-e2a4dccf3590 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:13.604237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea0420-9636-40ec-8f37-e2a4dccf3590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:13.613119 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:13.634652 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-77ea0420-9636-40ec-8f37-e2a4dccf3590 None None] MFA Rules not processed for user `01ec3acf20cc489da9dda278babb4e33`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:13.636012 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-77ea0420-9636-40ec-8f37-e2a4dccf3590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:13.702237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1207/2404] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:13.713759 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d1114-254f-427e-abaa-9896c9f3b197 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:13.715545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-5d0d1114-254f-427e-abaa-9896c9f3b197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:13.773859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d1114-254f-427e-abaa-9896c9f3b197 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:13.777634 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b43b16-52e2-478f-86ba-773fbcfd6112 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:13.778248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b43b16-52e2-478f-86ba-773fbcfd6112 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:13.778521 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b43b16-52e2-478f-86ba-773fbcfd6112 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:13.778789 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b43b16-52e2-478f-86ba-773fbcfd6112 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:13.784309 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:13.788220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b43b16-52e2-478f-86ba-773fbcfd6112 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:13.789791 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b43b16-52e2-478f-86ba-773fbcfd6112 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:13.800594 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1198/2405] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:13.822673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea0420-9636-40ec-8f37-e2a4dccf3590 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:13.886926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea0420-9636-40ec-8f37-e2a4dccf3590 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:13.891821 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac0fd1-ca0d-46ee-802d-45185a296dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:13.892490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac0fd1-ca0d-46ee-802d-45185a296dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:13.892773 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac0fd1-ca0d-46ee-802d-45185a296dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:13.893048 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac0fd1-ca0d-46ee-802d-45185a296dca admin admin] PATH_INFO: `/v3/users/01ec3acf20cc489da9dda278babb4e33` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:13.894318 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac0fd1-ca0d-46ee-802d-45185a296dca admin admin] RBAC: Authorizing `identity:delete_user(user_id=01ec3acf20cc489da9dda278babb4e33)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:13.896271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac0fd1-ca0d-46ee-802d-45185a296dca admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:13.958199 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a7ac0fd1-ca0d-46ee-802d-45185a296dca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01ec3acf20cc489da9dda278babb4e33'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:13.974716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a7ac0fd1-ca0d-46ee-802d-45185a296dca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01ec3acf20cc489da9dda278babb4e33'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:13.991981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a7ac0fd1-ca0d-46ee-802d-45185a296dca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01ec3acf20cc489da9dda278babb4e33'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:13.992781 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a7ac0fd1-ca0d-46ee-802d-45185a296dca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01ec3acf20cc489da9dda278babb4e33'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:13.993604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a7ac0fd1-ca0d-46ee-802d-45185a296dca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01ec3acf20cc489da9dda278babb4e33'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:14.013825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1208/2406] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:13 2022] DELETE /identity/v3/users/01ec3acf20cc489da9dda278babb4e33 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:14.020742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfea269-c499-49fe-bde2-be3ac022a68d None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:14.021025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfea269-c499-49fe-bde2-be3ac022a68d None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:14.021291 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfea269-c499-49fe-bde2-be3ac022a68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:14.036503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac0fd1-ca0d-46ee-802d-45185a296dca admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:14.038055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-a7ac0fd1-ca0d-46ee-802d-45185a296dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:14.044372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-ecfea269-c499-49fe-bde2-be3ac022a68d None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:14.045908 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-ecfea269-c499-49fe-bde2-be3ac022a68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:14.054525 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:14.108542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac0fd1-ca0d-46ee-802d-45185a296dca admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:14.112885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7d8c5-824b-4bc8-8680-d8e87610d75b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:14.113495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7d8c5-824b-4bc8-8680-d8e87610d75b service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:14.113775 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7d8c5-824b-4bc8-8680-d8e87610d75b service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:14.114113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7d8c5-824b-4bc8-8680-d8e87610d75b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:14.115757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-92c7d8c5-824b-4bc8-8680-d8e87610d75b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:14.116639 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1199/2407] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:14.127688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfea269-c499-49fe-bde2-be3ac022a68d None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:14.136569 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7d8c5-824b-4bc8-8680-d8e87610d75b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:14.138005 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfea269-c499-49fe-bde2-be3ac022a68d None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:14.138121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7d8c5-824b-4bc8-8680-d8e87610d75b service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:14.141890 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667b50b-c61f-4668-b8b0-568651fe357d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:14.142502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667b50b-c61f-4668-b8b0-568651fe357d service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:14.142823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667b50b-c61f-4668-b8b0-568651fe357d service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:14.143094 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667b50b-c61f-4668-b8b0-568651fe357d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:14.147787 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1209/2408] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:32:14.152972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667b50b-c61f-4668-b8b0-568651fe357d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:14.154648 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667b50b-c61f-4668-b8b0-568651fe357d service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:14.167067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1200/2409] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:14.647505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7d8c5-824b-4bc8-8680-d8e87610d75b service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:14.649229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-92c7d8c5-824b-4bc8-8680-d8e87610d75b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:14.662916 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7d8c5-824b-4bc8-8680-d8e87610d75b service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:14.668255 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0b1a4-0718-4af4-8e43-318f2b95f77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:14.668908 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0b1a4-0718-4af4-8e43-318f2b95f77c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:14.669190 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0b1a4-0718-4af4-8e43-318f2b95f77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:14.669461 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0b1a4-0718-4af4-8e43-318f2b95f77c admin admin] PATH_INFO: `/v3/projects/869cb17c79de4ebdbec6086f0e52f1e3` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:14.671072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0b1a4-0718-4af4-8e43-318f2b95f77c admin admin] RBAC: Authorizing `identity:delete_project(project_id=869cb17c79de4ebdbec6086f0e52f1e3)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:14.673094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0b1a4-0718-4af4-8e43-318f2b95f77c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:14.723528 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-84a0b1a4-0718-4af4-8e43-318f2b95f77c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '869cb17c79de4ebdbec6086f0e52f1e3', 'request_id': 'req-84a0b1a4-0718-4af4-8e43-318f2b95f77c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:14.729792 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-84a0b1a4-0718-4af4-8e43-318f2b95f77c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '869cb17c79de4ebdbec6086f0e52f1e3', 'request_id': 'req-84a0b1a4-0718-4af4-8e43-318f2b95f77c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:14.740253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-84a0b1a4-0718-4af4-8e43-318f2b95f77c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '869cb17c79de4ebdbec6086f0e52f1e3', 'request_id': 'req-84a0b1a4-0718-4af4-8e43-318f2b95f77c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:14.752191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-84a0b1a4-0718-4af4-8e43-318f2b95f77c admin admin] The token cache is being invalidate because project 869cb17c79de4ebdbec6086f0e52f1e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:14.752492 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-84a0b1a4-0718-4af4-8e43-318f2b95f77c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:14.753176 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-84a0b1a4-0718-4af4-8e43-318f2b95f77c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:14.754963 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1210/2410] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:14 2022] DELETE /identity/v3/projects/869cb17c79de4ebdbec6086f0e52f1e3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:14.760176 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b6037-d005-40ee-8b53-066f720e08d5 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:14.760729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b6037-d005-40ee-8b53-066f720e08d5 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:14.760729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b6037-d005-40ee-8b53-066f720e08d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:14.785837 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-e34b6037-d005-40ee-8b53-066f720e08d5 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:14.786996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e34b6037-d005-40ee-8b53-066f720e08d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:14.850706 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1201/2411] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:14.917849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d93ae-a1f7-46c7-827b-fed9534f1660 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:14.918324 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d93ae-a1f7-46c7-827b-fed9534f1660 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:14.918560 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d93ae-a1f7-46c7-827b-fed9534f1660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:14.941389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-f87d93ae-a1f7-46c7-827b-fed9534f1660 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:14.942765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f87d93ae-a1f7-46c7-827b-fed9534f1660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:14.951243 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1211/2412] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:14.956495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b6037-d005-40ee-8b53-066f720e08d5 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:14.966951 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b6037-d005-40ee-8b53-066f720e08d5 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:14.972239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46f401-5095-42a7-81a4-0bf42fcfa530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:14.972719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46f401-5095-42a7-81a4-0bf42fcfa530 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:14.972982 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46f401-5095-42a7-81a4-0bf42fcfa530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:14.973253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46f401-5095-42a7-81a4-0bf42fcfa530 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:14.973921 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46f401-5095-42a7-81a4-0bf42fcfa530 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:32:14.974223 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46f401-5095-42a7-81a4-0bf42fcfa530 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:14.975863 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46f401-5095-42a7-81a4-0bf42fcfa530 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:14.989250 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1202/2413] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:32:14.994126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d93ae-a1f7-46c7-827b-fed9534f1660 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:15.005143 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d93ae-a1f7-46c7-827b-fed9534f1660 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:15.016803 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d973de8-7747-492a-b821-ee85690670d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:15.017445 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d973de8-7747-492a-b821-ee85690670d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:15.017722 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d973de8-7747-492a-b821-ee85690670d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:15.017998 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d973de8-7747-492a-b821-ee85690670d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:15.018639 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d973de8-7747-492a-b821-ee85690670d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:15.020644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d973de8-7747-492a-b821-ee85690670d2 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:15.056169 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1212/2414] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:14 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:15.057637 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46f401-5095-42a7-81a4-0bf42fcfa530 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:15.068851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46f401-5095-42a7-81a4-0bf42fcfa530 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:15.148536 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1d420-eae3-4ff4-93f3-b59b3f961435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:15.149660 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1d420-eae3-4ff4-93f3-b59b3f961435 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:15.149930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1d420-eae3-4ff4-93f3-b59b3f961435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:15.150332 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1d420-eae3-4ff4-93f3-b59b3f961435 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:15.151290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c1d420-eae3-4ff4-93f3-b59b3f961435 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:15.154523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c1d420-eae3-4ff4-93f3-b59b3f961435 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:15.172694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:15.179626 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1203/2415] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:15 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:32:15.183230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d973de8-7747-492a-b821-ee85690670d2 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:15.194374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d973de8-7747-492a-b821-ee85690670d2 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:15.199094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717b117-bf33-4370-ac04-b56382197277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:15.199704 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717b117-bf33-4370-ac04-b56382197277 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:15.199985 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717b117-bf33-4370-ac04-b56382197277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:15.200303 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717b117-bf33-4370-ac04-b56382197277 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:15.200955 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2717b117-bf33-4370-ac04-b56382197277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:15.202556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2717b117-bf33-4370-ac04-b56382197277 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:15.214163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1213/2416] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:32:15.218398 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1d420-eae3-4ff4-93f3-b59b3f961435 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:15.229194 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1d420-eae3-4ff4-93f3-b59b3f961435 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:15.234533 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118dd1d4-7948-42b6-81f1-879bafdc5a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:15.235151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dd1d4-7948-42b6-81f1-879bafdc5a44 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:15.235435 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dd1d4-7948-42b6-81f1-879bafdc5a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:15.235707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118dd1d4-7948-42b6-81f1-879bafdc5a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:15.236499 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118dd1d4-7948-42b6-81f1-879bafdc5a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:15.238105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118dd1d4-7948-42b6-81f1-879bafdc5a44 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:15.249793 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1204/2417] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:15.254321 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717b117-bf33-4370-ac04-b56382197277 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:15.265257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2717b117-bf33-4370-ac04-b56382197277 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:15.270961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc4823-af81-43e9-afb2-e1349bc21dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:15.271705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc4823-af81-43e9-afb2-e1349bc21dfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:15.272010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc4823-af81-43e9-afb2-e1349bc21dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:15.272302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc4823-af81-43e9-afb2-e1349bc21dfd admin admin] PATH_INFO: `/v3/projects/dbdeded8a9b041e999e1997f7db4b942/users/fd012b2dc59240ce97f5c44fcf06e789/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:15.287695 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc4823-af81-43e9-afb2-e1349bc21dfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbdeded8a9b041e999e1997f7db4b942, user_id=fd012b2dc59240ce97f5c44fcf06e789, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:15.290380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc4823-af81-43e9-afb2-e1349bc21dfd admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:15.305664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1214/2418] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:15 2022] PUT /identity/v3/projects/dbdeded8a9b041e999e1997f7db4b942/users/fd012b2dc59240ce97f5c44fcf06e789/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:15.321340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19769bfa-63ad-4aa3-ad24-fcd0978f4f55 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:15.321576 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19769bfa-63ad-4aa3-ad24-fcd0978f4f55 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:15.321881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19769bfa-63ad-4aa3-ad24-fcd0978f4f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:15.349670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-19769bfa-63ad-4aa3-ad24-fcd0978f4f55 None None] MFA Rules not processed for user `fd012b2dc59240ce97f5c44fcf06e789`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:15.350892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-19769bfa-63ad-4aa3-ad24-fcd0978f4f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:15.422056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1205/2419] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:15.427294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc4823-af81-43e9-afb2-e1349bc21dfd admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:15.475892 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:15.501513 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc4823-af81-43e9-afb2-e1349bc21dfd admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:15.506601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa295bb-7120-41f8-a68c-a04b333a4e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:15.507339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa295bb-7120-41f8-a68c-a04b333a4e64 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:15.507706 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa295bb-7120-41f8-a68c-a04b333a4e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:15.508294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa295bb-7120-41f8-a68c-a04b333a4e64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:15.508813 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa295bb-7120-41f8-a68c-a04b333a4e64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:15.510854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa295bb-7120-41f8-a68c-a04b333a4e64 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:15.544294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1215/2420] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:15 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:15.549239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19769bfa-63ad-4aa3-ad24-fcd0978f4f55 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:15.563634 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19769bfa-63ad-4aa3-ad24-fcd0978f4f55 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:15.627857 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78e7c2-ba23-407a-8048-1a7e4264fa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:15.628589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78e7c2-ba23-407a-8048-1a7e4264fa8d admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:15.628875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78e7c2-ba23-407a-8048-1a7e4264fa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:15.629147 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78e7c2-ba23-407a-8048-1a7e4264fa8d admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:15.629787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78e7c2-ba23-407a-8048-1a7e4264fa8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:15.631774 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78e7c2-ba23-407a-8048-1a7e4264fa8d admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:15.655624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1206/2421] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:15 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:32:15.660172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa295bb-7120-41f8-a68c-a04b333a4e64 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:15.672189 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa295bb-7120-41f8-a68c-a04b333a4e64 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:15.677291 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f464bf-99f3-446e-83f9-8c97ed92eee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:15.678009 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f464bf-99f3-446e-83f9-8c97ed92eee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:15.678510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f464bf-99f3-446e-83f9-8c97ed92eee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:15.678857 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f464bf-99f3-446e-83f9-8c97ed92eee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:15.679590 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f464bf-99f3-446e-83f9-8c97ed92eee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:15.681487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f464bf-99f3-446e-83f9-8c97ed92eee8 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:15.694455 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1216/2422] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:15.698767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78e7c2-ba23-407a-8048-1a7e4264fa8d admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:15.710613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78e7c2-ba23-407a-8048-1a7e4264fa8d admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:15.715259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c50dbe-1fea-49af-9f4c-a04eecc4c6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:15.715922 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c50dbe-1fea-49af-9f4c-a04eecc4c6be admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:15.716229 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c50dbe-1fea-49af-9f4c-a04eecc4c6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:15.716500 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c50dbe-1fea-49af-9f4c-a04eecc4c6be admin admin] PATH_INFO: `/v3/projects/c65b27781e604f04b56b29845e8e2654/users/d5f2db4ccbed4c188c1e657de96abf5a/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:15.732514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c50dbe-1fea-49af-9f4c-a04eecc4c6be admin admin] RBAC: Authorizing `identity:create_grant(project_id=c65b27781e604f04b56b29845e8e2654, user_id=d5f2db4ccbed4c188c1e657de96abf5a, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:15.735180 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c50dbe-1fea-49af-9f4c-a04eecc4c6be admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:15.753916 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1207/2423] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:15 2022] PUT /identity/v3/projects/c65b27781e604f04b56b29845e8e2654/users/d5f2db4ccbed4c188c1e657de96abf5a/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:32:15.758540 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f464bf-99f3-446e-83f9-8c97ed92eee8 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:15.770923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f464bf-99f3-446e-83f9-8c97ed92eee8 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:15.830892 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8eda6-c05e-4a79-b763-478fa663d422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:15.831511 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8eda6-c05e-4a79-b763-478fa663d422 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:15.831830 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8eda6-c05e-4a79-b763-478fa663d422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:15.832146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8eda6-c05e-4a79-b763-478fa663d422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:15.832811 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8eda6-c05e-4a79-b763-478fa663d422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:15.834399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8eda6-c05e-4a79-b763-478fa663d422 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:15.844551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1217/2424] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:15.848635 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c50dbe-1fea-49af-9f4c-a04eecc4c6be admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:15.859023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c50dbe-1fea-49af-9f4c-a04eecc4c6be admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:15.863902 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aee6b4-b35e-4f7b-9124-62aedd757b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:15.864586 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aee6b4-b35e-4f7b-9124-62aedd757b82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:15.864864 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aee6b4-b35e-4f7b-9124-62aedd757b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:15.865136 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aee6b4-b35e-4f7b-9124-62aedd757b82 admin admin] PATH_INFO: `/v3/projects/c65b27781e604f04b56b29845e8e2654/users/d5f2db4ccbed4c188c1e657de96abf5a/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:15.867041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aee6b4-b35e-4f7b-9124-62aedd757b82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c65b27781e604f04b56b29845e8e2654, user_id=d5f2db4ccbed4c188c1e657de96abf5a, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:15.869729 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aee6b4-b35e-4f7b-9124-62aedd757b82 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:15.880876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1208/2425] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:15 2022] PUT /identity/v3/projects/c65b27781e604f04b56b29845e8e2654/users/d5f2db4ccbed4c188c1e657de96abf5a/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:15.892416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c325b-9a78-4599-8940-1f8c3f1c70cd None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:15.892676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c325b-9a78-4599-8940-1f8c3f1c70cd None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:15.892937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c325b-9a78-4599-8940-1f8c3f1c70cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:15.922456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-f05c325b-9a78-4599-8940-1f8c3f1c70cd None None] MFA Rules not processed for user `d5f2db4ccbed4c188c1e657de96abf5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:15.923589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f05c325b-9a78-4599-8940-1f8c3f1c70cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:16.000651 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1218/2426] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:32:16.017987 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aee6b4-b35e-4f7b-9124-62aedd757b82 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:16.019804 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-84aee6b4-b35e-4f7b-9124-62aedd757b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:16.056022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:16.089677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aee6b4-b35e-4f7b-9124-62aedd757b82 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:16.093575 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9f75a-7f2d-4a6d-8539-0c6af00e232f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:16.094197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9f75a-7f2d-4a6d-8539-0c6af00e232f service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:16.094472 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9f75a-7f2d-4a6d-8539-0c6af00e232f service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:16.094738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9f75a-7f2d-4a6d-8539-0c6af00e232f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:16.162355 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9f75a-7f2d-4a6d-8539-0c6af00e232f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:16.163824 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9f75a-7f2d-4a6d-8539-0c6af00e232f service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:16.174739 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1209/2427] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:17.683331 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c325b-9a78-4599-8940-1f8c3f1c70cd None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:17.684837 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f05c325b-9a78-4599-8940-1f8c3f1c70cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:17.721716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:17.768021 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c325b-9a78-4599-8940-1f8c3f1c70cd None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:17.776760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8a951-4e7b-4cb0-85b1-44ffed4188fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:17.777570 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8a951-4e7b-4cb0-85b1-44ffed4188fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:17.777969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8a951-4e7b-4cb0-85b1-44ffed4188fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:17.778451 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8a951-4e7b-4cb0-85b1-44ffed4188fa admin admin] PATH_INFO: `/v3/users/6732487041c54918b1164b1cb72bb07e` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:17.779699 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8a951-4e7b-4cb0-85b1-44ffed4188fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6732487041c54918b1164b1cb72bb07e)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:17.782105 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8a951-4e7b-4cb0-85b1-44ffed4188fa admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:17.839870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-d0c8a951-4e7b-4cb0-85b1-44ffed4188fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6732487041c54918b1164b1cb72bb07e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:17.853118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-d0c8a951-4e7b-4cb0-85b1-44ffed4188fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6732487041c54918b1164b1cb72bb07e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:17.860451 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-d0c8a951-4e7b-4cb0-85b1-44ffed4188fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6732487041c54918b1164b1cb72bb07e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:17.861213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-d0c8a951-4e7b-4cb0-85b1-44ffed4188fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6732487041c54918b1164b1cb72bb07e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:17.861911 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-d0c8a951-4e7b-4cb0-85b1-44ffed4188fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6732487041c54918b1164b1cb72bb07e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:17.888735 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1219/2428] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:17 2022] DELETE /identity/v3/users/6732487041c54918b1164b1cb72bb07e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:18.094227 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9f75a-7f2d-4a6d-8539-0c6af00e232f service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:18.115633 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9f75a-7f2d-4a6d-8539-0c6af00e232f service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:18.181190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63e22e-3c55-47fd-9f25-b4de58e4cfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:18.181879 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63e22e-3c55-47fd-9f25-b4de58e4cfd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:18.182157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63e22e-3c55-47fd-9f25-b4de58e4cfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:18.182431 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63e22e-3c55-47fd-9f25-b4de58e4cfd4 admin admin] PATH_INFO: `/v3/projects/fcf93b388fac4be38661d185a7231937` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:18.183594 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63e22e-3c55-47fd-9f25-b4de58e4cfd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcf93b388fac4be38661d185a7231937)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:18.185559 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63e22e-3c55-47fd-9f25-b4de58e4cfd4 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:18.236505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5e63e22e-3c55-47fd-9f25-b4de58e4cfd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcf93b388fac4be38661d185a7231937', 'request_id': 'req-5e63e22e-3c55-47fd-9f25-b4de58e4cfd4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:18.242545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5e63e22e-3c55-47fd-9f25-b4de58e4cfd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcf93b388fac4be38661d185a7231937', 'request_id': 'req-5e63e22e-3c55-47fd-9f25-b4de58e4cfd4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:18.249003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5e63e22e-3c55-47fd-9f25-b4de58e4cfd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcf93b388fac4be38661d185a7231937', 'request_id': 'req-5e63e22e-3c55-47fd-9f25-b4de58e4cfd4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:18.262708 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5e63e22e-3c55-47fd-9f25-b4de58e4cfd4 admin admin] The token cache is being invalidate because project fcf93b388fac4be38661d185a7231937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:18.262993 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5e63e22e-3c55-47fd-9f25-b4de58e4cfd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:18.263601 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-5e63e22e-3c55-47fd-9f25-b4de58e4cfd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:18.265191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1210/2429] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:18 2022] DELETE /identity/v3/projects/fcf93b388fac4be38661d185a7231937 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:18.270446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7a6fd-5d91-4945-8a27-529bedcba49e None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:18.270704 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7a6fd-5d91-4945-8a27-529bedcba49e None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:18.270978 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7a6fd-5d91-4945-8a27-529bedcba49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:18.292076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-fac7a6fd-5d91-4945-8a27-529bedcba49e None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:18.293302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-fac7a6fd-5d91-4945-8a27-529bedcba49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:18.356796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1220/2430] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:18.367219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7391ba-755a-4355-ac6f-7c1a0ef3f77b None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:18.367505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7391ba-755a-4355-ac6f-7c1a0ef3f77b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:18.367783 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7391ba-755a-4355-ac6f-7c1a0ef3f77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:18.395267 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-8c7391ba-755a-4355-ac6f-7c1a0ef3f77b None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:18.396520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8c7391ba-755a-4355-ac6f-7c1a0ef3f77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:18.408091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1211/2431] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:18.413663 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7a6fd-5d91-4945-8a27-529bedcba49e None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:18.428748 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7a6fd-5d91-4945-8a27-529bedcba49e None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:18.434928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734f455-1e28-4758-ae58-05fbd8c261b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:18.435550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734f455-1e28-4758-ae58-05fbd8c261b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:18.435830 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734f455-1e28-4758-ae58-05fbd8c261b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:18.436139 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734f455-1e28-4758-ae58-05fbd8c261b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:18.436800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734f455-1e28-4758-ae58-05fbd8c261b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:32:18.437106 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734f455-1e28-4758-ae58-05fbd8c261b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:18.438705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734f455-1e28-4758-ae58-05fbd8c261b2 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:18.451437 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1221/2432] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:32:18.456046 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7391ba-755a-4355-ac6f-7c1a0ef3f77b None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:18.466046 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7391ba-755a-4355-ac6f-7c1a0ef3f77b None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:18.474316 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477be073-bb04-4353-916b-d4f41fc53ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:18.475181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477be073-bb04-4353-916b-d4f41fc53ec8 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:18.475523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477be073-bb04-4353-916b-d4f41fc53ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:18.475887 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477be073-bb04-4353-916b-d4f41fc53ec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:18.476785 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477be073-bb04-4353-916b-d4f41fc53ec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:18.479757 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477be073-bb04-4353-916b-d4f41fc53ec8 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:18.486684 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:18.512019 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1212/2433] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:18 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:18.516229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734f455-1e28-4758-ae58-05fbd8c261b2 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:18.526820 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734f455-1e28-4758-ae58-05fbd8c261b2 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:18.567617 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:18.592673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe567a-3ee1-4795-966d-0e9f80fd8063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:18.593283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe567a-3ee1-4795-966d-0e9f80fd8063 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:18.593558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe567a-3ee1-4795-966d-0e9f80fd8063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:18.593830 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe567a-3ee1-4795-966d-0e9f80fd8063 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:18.594450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe567a-3ee1-4795-966d-0e9f80fd8063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:18.596399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe567a-3ee1-4795-966d-0e9f80fd8063 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:18.620079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1222/2434] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:18 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:18.624344 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477be073-bb04-4353-916b-d4f41fc53ec8 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:18.635022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477be073-bb04-4353-916b-d4f41fc53ec8 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:18.639614 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ab7d1-a9a6-48b2-985e-acdf5c170f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:18.640256 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ab7d1-a9a6-48b2-985e-acdf5c170f68 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:18.640531 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ab7d1-a9a6-48b2-985e-acdf5c170f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:18.640801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ab7d1-a9a6-48b2-985e-acdf5c170f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:18.641447 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ab7d1-a9a6-48b2-985e-acdf5c170f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:18.643057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ab7d1-a9a6-48b2-985e-acdf5c170f68 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:18.653286 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1213/2435] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:18.657482 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe567a-3ee1-4795-966d-0e9f80fd8063 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:18.668031 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe567a-3ee1-4795-966d-0e9f80fd8063 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:18.673453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ee85e-44d1-4c96-9824-1205ed4e5ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:18.674051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ee85e-44d1-4c96-9824-1205ed4e5ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:18.674334 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ee85e-44d1-4c96-9824-1205ed4e5ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:18.674606 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ee85e-44d1-4c96-9824-1205ed4e5ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:18.675255 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ee85e-44d1-4c96-9824-1205ed4e5ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:18.676908 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ee85e-44d1-4c96-9824-1205ed4e5ec0 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:18.686887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1223/2436] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:18.691039 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ab7d1-a9a6-48b2-985e-acdf5c170f68 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:18.705011 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ab7d1-a9a6-48b2-985e-acdf5c170f68 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:18.710201 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745d7d8-ef95-43c7-9fc7-c71299c1858d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:18.710855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745d7d8-ef95-43c7-9fc7-c71299c1858d admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:18.711134 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745d7d8-ef95-43c7-9fc7-c71299c1858d admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:18.711406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745d7d8-ef95-43c7-9fc7-c71299c1858d admin admin] PATH_INFO: `/v3/projects/b629d01691534ffa8bb750edfd4df96d/users/787de57b64a84860bcad23ad72800b0e/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:18.726982 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745d7d8-ef95-43c7-9fc7-c71299c1858d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b629d01691534ffa8bb750edfd4df96d, user_id=787de57b64a84860bcad23ad72800b0e, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:18.729710 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745d7d8-ef95-43c7-9fc7-c71299c1858d admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:18.744984 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1214/2437] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:18 2022] PUT /identity/v3/projects/b629d01691534ffa8bb750edfd4df96d/users/787de57b64a84860bcad23ad72800b0e/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:18.757415 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0b752-1631-4129-86c5-a8fee88b8cc3 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:18.757700 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0b752-1631-4129-86c5-a8fee88b8cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:18.757970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0b752-1631-4129-86c5-a8fee88b8cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:18.790046 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-20b0b752-1631-4129-86c5-a8fee88b8cc3 None None] MFA Rules not processed for user `787de57b64a84860bcad23ad72800b0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:18.790538 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-20b0b752-1631-4129-86c5-a8fee88b8cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:18.813276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745d7d8-ef95-43c7-9fc7-c71299c1858d admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:18.815714 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8745d7d8-ef95-43c7-9fc7-c71299c1858d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:18.867649 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1224/2438] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:18.879364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0b752-1631-4129-86c5-a8fee88b8cc3 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:18.881080 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-20b0b752-1631-4129-86c5-a8fee88b8cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:18.890309 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745d7d8-ef95-43c7-9fc7-c71299c1858d admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:18.895317 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8d2bc5-2d98-4a95-9842-909a744ce710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:18.895982 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d2bc5-2d98-4a95-9842-909a744ce710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:18.896296 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d2bc5-2d98-4a95-9842-909a744ce710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:18.896561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8d2bc5-2d98-4a95-9842-909a744ce710 admin admin] PATH_INFO: `/v3/users/fd012b2dc59240ce97f5c44fcf06e789` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:18.897791 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d2bc5-2d98-4a95-9842-909a744ce710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd012b2dc59240ce97f5c44fcf06e789)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:18.899764 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8d2bc5-2d98-4a95-9842-909a744ce710 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:18.948495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7c8d2bc5-2d98-4a95-9842-909a744ce710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd012b2dc59240ce97f5c44fcf06e789'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:18.955309 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7c8d2bc5-2d98-4a95-9842-909a744ce710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd012b2dc59240ce97f5c44fcf06e789'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:18.957108 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0b752-1631-4129-86c5-a8fee88b8cc3 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:18.962726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37662817-0a6d-4782-8cab-8442cba508ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:18.963340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37662817-0a6d-4782-8cab-8442cba508ca service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:18.963621 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37662817-0a6d-4782-8cab-8442cba508ca service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:18.963889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37662817-0a6d-4782-8cab-8442cba508ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:18.968098 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7c8d2bc5-2d98-4a95-9842-909a744ce710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd012b2dc59240ce97f5c44fcf06e789'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:18.968812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7c8d2bc5-2d98-4a95-9842-909a744ce710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd012b2dc59240ce97f5c44fcf06e789'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:18.972727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37662817-0a6d-4782-8cab-8442cba508ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:18.975110 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37662817-0a6d-4782-8cab-8442cba508ca service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:18.978817 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7c8d2bc5-2d98-4a95-9842-909a744ce710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd012b2dc59240ce97f5c44fcf06e789'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:18.985079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1215/2439] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:18 2022] DELETE /identity/v3/users/fd012b2dc59240ce97f5c44fcf06e789 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:18.988943 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1225/2440] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:18.989735 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953f38f-bbb2-48e8-869b-97f1f9667631 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:18.990040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953f38f-bbb2-48e8-869b-97f1f9667631 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:18.990312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953f38f-bbb2-48e8-869b-97f1f9667631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:19.011105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-b953f38f-bbb2-48e8-869b-97f1f9667631 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:19.011105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b953f38f-bbb2-48e8-869b-97f1f9667631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:19.039420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:19.077129 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1216/2441] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:32:19.086906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37662817-0a6d-4782-8cab-8442cba508ca service nova] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:19.088374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-37662817-0a6d-4782-8cab-8442cba508ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:19.134213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:19.166243 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37662817-0a6d-4782-8cab-8442cba508ca service nova] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:19.212814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953f38f-bbb2-48e8-869b-97f1f9667631 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:19.215829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b953f38f-bbb2-48e8-869b-97f1f9667631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:19.218980 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ccf5a-4054-4607-8b41-b7a987d7c936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:19.219603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ccf5a-4054-4607-8b41-b7a987d7c936 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:19.219939 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ccf5a-4054-4607-8b41-b7a987d7c936 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:19.222200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ccf5a-4054-4607-8b41-b7a987d7c936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:19.228410 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953f38f-bbb2-48e8-869b-97f1f9667631 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:19.235084 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ccf5a-4054-4607-8b41-b7a987d7c936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:19.237000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ccf5a-4054-4607-8b41-b7a987d7c936 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:19.237309 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209058e2-845d-41e0-ac54-6c270c0f1450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:19.238061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209058e2-845d-41e0-ac54-6c270c0f1450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:19.238334 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209058e2-845d-41e0-ac54-6c270c0f1450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:19.238602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209058e2-845d-41e0-ac54-6c270c0f1450 admin admin] PATH_INFO: `/v3/users/787de57b64a84860bcad23ad72800b0e` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:19.239754 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209058e2-845d-41e0-ac54-6c270c0f1450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=787de57b64a84860bcad23ad72800b0e)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:19.241763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209058e2-845d-41e0-ac54-6c270c0f1450 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:19.248040 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1226/2442] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:19.291265 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-209058e2-845d-41e0-ac54-6c270c0f1450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '787de57b64a84860bcad23ad72800b0e'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:19.299573 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-209058e2-845d-41e0-ac54-6c270c0f1450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '787de57b64a84860bcad23ad72800b0e'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:19.312875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-209058e2-845d-41e0-ac54-6c270c0f1450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '787de57b64a84860bcad23ad72800b0e'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:19.313736 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-209058e2-845d-41e0-ac54-6c270c0f1450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '787de57b64a84860bcad23ad72800b0e'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:19.321877 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-209058e2-845d-41e0-ac54-6c270c0f1450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '787de57b64a84860bcad23ad72800b0e'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:19.328131 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1217/2443] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:19 2022] DELETE /identity/v3/users/787de57b64a84860bcad23ad72800b0e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:32:19.332624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c614b2-1dfc-4b9f-9e36-978a71b8551f None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:19.332914 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c614b2-1dfc-4b9f-9e36-978a71b8551f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:19.333191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c614b2-1dfc-4b9f-9e36-978a71b8551f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:19.352756 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-67c614b2-1dfc-4b9f-9e36-978a71b8551f None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:19.353979 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-67c614b2-1dfc-4b9f-9e36-978a71b8551f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:19.418970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1227/2444] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:32:19.429307 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209058e2-845d-41e0-ac54-6c270c0f1450 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:19.431201 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-209058e2-845d-41e0-ac54-6c270c0f1450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:19.490192 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209058e2-845d-41e0-ac54-6c270c0f1450 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:19.493968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a367b-ee07-49c2-ba0e-c6b903281ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:19.494633 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a367b-ee07-49c2-ba0e-c6b903281ffc service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:19.495121 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a367b-ee07-49c2-ba0e-c6b903281ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:19.495366 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a367b-ee07-49c2-ba0e-c6b903281ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:19.508162 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a367b-ee07-49c2-ba0e-c6b903281ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:19.509792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a367b-ee07-49c2-ba0e-c6b903281ffc service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:19.520643 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c614b2-1dfc-4b9f-9e36-978a71b8551f None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:19.522513 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-67c614b2-1dfc-4b9f-9e36-978a71b8551f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:19.525879 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1218/2445] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:19.535745 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c614b2-1dfc-4b9f-9e36-978a71b8551f None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:19.541396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:19.542087 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:19.542369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:19.542709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c admin admin] PATH_INFO: `/v3/projects/dbdeded8a9b041e999e1997f7db4b942` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:19.543978 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbdeded8a9b041e999e1997f7db4b942)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:19.546501 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:19.601195 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbdeded8a9b041e999e1997f7db4b942', 'request_id': 'req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:19.607748 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbdeded8a9b041e999e1997f7db4b942', 'request_id': 'req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:19.616060 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbdeded8a9b041e999e1997f7db4b942', 'request_id': 'req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:19.627751 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c admin admin] The token cache is being invalidate because project dbdeded8a9b041e999e1997f7db4b942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:19.628118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:19.628855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:19.630842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1228/2446] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:19 2022] DELETE /identity/v3/projects/dbdeded8a9b041e999e1997f7db4b942 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:19.635126 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a367b-ee07-49c2-ba0e-c6b903281ffc service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:19.645208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a367b-ee07-49c2-ba0e-c6b903281ffc service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:19.691365 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:19.708606 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3e5b5-0104-4921-95ce-771c480ac0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:19.709380 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3e5b5-0104-4921-95ce-771c480ac0bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:19.709723 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3e5b5-0104-4921-95ce-771c480ac0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:19.710057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3e5b5-0104-4921-95ce-771c480ac0bc admin admin] PATH_INFO: `/v3/users/d5f2db4ccbed4c188c1e657de96abf5a` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:19.711289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3e5b5-0104-4921-95ce-771c480ac0bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5f2db4ccbed4c188c1e657de96abf5a)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:19.713490 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3e5b5-0104-4921-95ce-771c480ac0bc admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:19.765243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-66e3e5b5-0104-4921-95ce-771c480ac0bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5f2db4ccbed4c188c1e657de96abf5a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:19.772271 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-66e3e5b5-0104-4921-95ce-771c480ac0bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5f2db4ccbed4c188c1e657de96abf5a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:19.784545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-66e3e5b5-0104-4921-95ce-771c480ac0bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5f2db4ccbed4c188c1e657de96abf5a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:19.785317 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-66e3e5b5-0104-4921-95ce-771c480ac0bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5f2db4ccbed4c188c1e657de96abf5a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:19.793212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-66e3e5b5-0104-4921-95ce-771c480ac0bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5f2db4ccbed4c188c1e657de96abf5a'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:19.800487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1219/2447] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:19 2022] DELETE /identity/v3/users/d5f2db4ccbed4c188c1e657de96abf5a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:19.837082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:19.838635 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:19.852551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23ead3-dab2-44f3-b1c5-62754cc8f51c admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:19.875978 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:19.935731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd66de4-be66-4aec-8851-aa8e870a1f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:19.936435 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd66de4-be66-4aec-8851-aa8e870a1f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:19.936718 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd66de4-be66-4aec-8851-aa8e870a1f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:19.937015 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd66de4-be66-4aec-8851-aa8e870a1f6e admin admin] PATH_INFO: `/v3/projects/b629d01691534ffa8bb750edfd4df96d` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:19.938186 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd66de4-be66-4aec-8851-aa8e870a1f6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b629d01691534ffa8bb750edfd4df96d)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:19.940176 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd66de4-be66-4aec-8851-aa8e870a1f6e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:19.992112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-6bd66de4-be66-4aec-8851-aa8e870a1f6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b629d01691534ffa8bb750edfd4df96d', 'request_id': 'req-6bd66de4-be66-4aec-8851-aa8e870a1f6e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:19.998452 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-6bd66de4-be66-4aec-8851-aa8e870a1f6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b629d01691534ffa8bb750edfd4df96d', 'request_id': 'req-6bd66de4-be66-4aec-8851-aa8e870a1f6e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:20.007173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-6bd66de4-be66-4aec-8851-aa8e870a1f6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b629d01691534ffa8bb750edfd4df96d', 'request_id': 'req-6bd66de4-be66-4aec-8851-aa8e870a1f6e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:20.018964 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-6bd66de4-be66-4aec-8851-aa8e870a1f6e admin admin] The token cache is being invalidate because project b629d01691534ffa8bb750edfd4df96d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:20.019339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-6bd66de4-be66-4aec-8851-aa8e870a1f6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:20.020084 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-6bd66de4-be66-4aec-8851-aa8e870a1f6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:20.021622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1229/2448] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:19 2022] DELETE /identity/v3/projects/b629d01691534ffa8bb750edfd4df96d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:20.026522 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92469f3-2b12-4caa-81e6-2a91888600ed None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.026884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92469f3-2b12-4caa-81e6-2a91888600ed None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.027228 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92469f3-2b12-4caa-81e6-2a91888600ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.049259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-b92469f3-2b12-4caa-81e6-2a91888600ed None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:20.050810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b92469f3-2b12-4caa-81e6-2a91888600ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:20.101086 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd66de4-be66-4aec-8851-aa8e870a1f6e admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.102515 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6bd66de4-be66-4aec-8851-aa8e870a1f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:20.116900 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1220/2449] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:20.124953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a08d2-920f-4f58-8ccd-7710dcf67521 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.125220 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a08d2-920f-4f58-8ccd-7710dcf67521 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.125568 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a08d2-920f-4f58-8ccd-7710dcf67521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.147060 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-6e4a08d2-920f-4f58-8ccd-7710dcf67521 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:20.148310 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-6e4a08d2-920f-4f58-8ccd-7710dcf67521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:20.156204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1221/2450] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:20.160468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a08d2-920f-4f58-8ccd-7710dcf67521 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.171274 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a08d2-920f-4f58-8ccd-7710dcf67521 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:20.175896 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f139902-1743-478b-8839-ecfcb183ee2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:20.176528 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f139902-1743-478b-8839-ecfcb183ee2e admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.176800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f139902-1743-478b-8839-ecfcb183ee2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.177067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f139902-1743-478b-8839-ecfcb183ee2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.177164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd66de4-be66-4aec-8851-aa8e870a1f6e admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:20.177746 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f139902-1743-478b-8839-ecfcb183ee2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:32:20.178074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f139902-1743-478b-8839-ecfcb183ee2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:20.179676 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f139902-1743-478b-8839-ecfcb183ee2e admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:20.182154 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99f798-8f9b-4846-b674-6180087a6016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:20.182862 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99f798-8f9b-4846-b674-6180087a6016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.183140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99f798-8f9b-4846-b674-6180087a6016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.183417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99f798-8f9b-4846-b674-6180087a6016 admin admin] PATH_INFO: `/v3/projects/c65b27781e604f04b56b29845e8e2654` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.184582 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99f798-8f9b-4846-b674-6180087a6016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c65b27781e604f04b56b29845e8e2654)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:20.186463 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99f798-8f9b-4846-b674-6180087a6016 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:20.192021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1222/2451] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:32:20.196326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f139902-1743-478b-8839-ecfcb183ee2e admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.205757 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f139902-1743-478b-8839-ecfcb183ee2e admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:20.210297 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2063223-97ee-4a63-8004-86fb78d57e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:20.211085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2063223-97ee-4a63-8004-86fb78d57e4a admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.211211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2063223-97ee-4a63-8004-86fb78d57e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.211455 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2063223-97ee-4a63-8004-86fb78d57e4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.212035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2063223-97ee-4a63-8004-86fb78d57e4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:20.214231 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2063223-97ee-4a63-8004-86fb78d57e4a admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:20.233193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:20.236612 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0f99f798-8f9b-4846-b674-6180087a6016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c65b27781e604f04b56b29845e8e2654', 'request_id': 'req-0f99f798-8f9b-4846-b674-6180087a6016'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:20.244305 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1223/2452] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:20 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:32:20.244516 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0f99f798-8f9b-4846-b674-6180087a6016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c65b27781e604f04b56b29845e8e2654', 'request_id': 'req-0f99f798-8f9b-4846-b674-6180087a6016'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:20.251169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2063223-97ee-4a63-8004-86fb78d57e4a admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.264275 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0f99f798-8f9b-4846-b674-6180087a6016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c65b27781e604f04b56b29845e8e2654', 'request_id': 'req-0f99f798-8f9b-4846-b674-6180087a6016'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:20.275321 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0f99f798-8f9b-4846-b674-6180087a6016 admin admin] The token cache is being invalidate because project c65b27781e604f04b56b29845e8e2654 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:20.275609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0f99f798-8f9b-4846-b674-6180087a6016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:20.276286 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0f99f798-8f9b-4846-b674-6180087a6016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:20.277838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1230/2453] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:20 2022] DELETE /identity/v3/projects/c65b27781e604f04b56b29845e8e2654 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:32:20.283330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71287bf8-1a47-4ffe-9e5c-9e51d04cafb0 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.283609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71287bf8-1a47-4ffe-9e5c-9e51d04cafb0 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.283877 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71287bf8-1a47-4ffe-9e5c-9e51d04cafb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.285527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:20.305313 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-71287bf8-1a47-4ffe-9e5c-9e51d04cafb0 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:20.306791 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-71287bf8-1a47-4ffe-9e5c-9e51d04cafb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:20.326049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2063223-97ee-4a63-8004-86fb78d57e4a admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:20.331434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3c2bb-cc6a-470d-baa4-aa0afb808872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:20.332711 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3c2bb-cc6a-470d-baa4-aa0afb808872 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.333239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3c2bb-cc6a-470d-baa4-aa0afb808872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.333530 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3c2bb-cc6a-470d-baa4-aa0afb808872 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.334575 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3c2bb-cc6a-470d-baa4-aa0afb808872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:20.336679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3c2bb-cc6a-470d-baa4-aa0afb808872 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:20.359833 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1224/2454] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:20 2022] POST /identity/v3/users => generated 302 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:32:20.363956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3c2bb-cc6a-470d-baa4-aa0afb808872 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.365847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-fcc3c2bb-cc6a-470d-baa4-aa0afb808872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:20.370438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1231/2455] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:20.378071 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3c2bb-cc6a-470d-baa4-aa0afb808872 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:20.378338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719a5ab-a3f7-4794-a8ff-15da246466f9 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.378743 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719a5ab-a3f7-4794-a8ff-15da246466f9 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.379007 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719a5ab-a3f7-4794-a8ff-15da246466f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.383043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a4589-b4c3-467c-8f6b-ea93d8b60db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:20.383532 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a4589-b4c3-467c-8f6b-ea93d8b60db5 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.383811 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a4589-b4c3-467c-8f6b-ea93d8b60db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.384119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a4589-b4c3-467c-8f6b-ea93d8b60db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.384765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a4589-b4c3-467c-8f6b-ea93d8b60db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:20.386709 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a4589-b4c3-467c-8f6b-ea93d8b60db5 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:20.397573 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1225/2456] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:20.400371 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-3719a5ab-a3f7-4794-a8ff-15da246466f9 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:20.401504 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a4589-b4c3-467c-8f6b-ea93d8b60db5 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.401883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-3719a5ab-a3f7-4794-a8ff-15da246466f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:20.410379 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1232/2457] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:32:20.412285 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a4589-b4c3-467c-8f6b-ea93d8b60db5 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:20.414761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719a5ab-a3f7-4794-a8ff-15da246466f9 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.417112 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d063b-cc9c-4532-b1a1-756459ceeef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:20.417731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d063b-cc9c-4532-b1a1-756459ceeef6 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.418009 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d063b-cc9c-4532-b1a1-756459ceeef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.418281 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d063b-cc9c-4532-b1a1-756459ceeef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.418934 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d063b-cc9c-4532-b1a1-756459ceeef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:20.420998 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d063b-cc9c-4532-b1a1-756459ceeef6 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:20.425796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719a5ab-a3f7-4794-a8ff-15da246466f9 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:20.430404 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350a603-713d-4b29-86d6-b48f3491f8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:20.430972 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350a603-713d-4b29-86d6-b48f3491f8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.431248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350a603-713d-4b29-86d6-b48f3491f8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.431520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350a603-713d-4b29-86d6-b48f3491f8d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.432232 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350a603-713d-4b29-86d6-b48f3491f8d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:32:20.432386 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1226/2458] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:32:20.432546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350a603-713d-4b29-86d6-b48f3491f8d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:20.434127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b350a603-713d-4b29-86d6-b48f3491f8d3 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:20.436771 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d063b-cc9c-4532-b1a1-756459ceeef6 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.448025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1233/2459] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:32:20.448964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d063b-cc9c-4532-b1a1-756459ceeef6 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:20.451104 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350a603-713d-4b29-86d6-b48f3491f8d3 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.457542 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4a33d-8875-481e-b66b-87095b57eac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:20.458213 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4a33d-8875-481e-b66b-87095b57eac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.458487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4a33d-8875-481e-b66b-87095b57eac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.458753 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4a33d-8875-481e-b66b-87095b57eac5 admin admin] PATH_INFO: `/v3/projects/6e17bb7af1b249d5870c85fd6ff183bc/users/ccb0f2ac006245eb87e26a9672952f66/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.460793 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b350a603-713d-4b29-86d6-b48f3491f8d3 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:20.465387 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d8b2a-748e-4988-8d29-07ab0e2636e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:20.465971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d8b2a-748e-4988-8d29-07ab0e2636e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.466248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d8b2a-748e-4988-8d29-07ab0e2636e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.466504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d8b2a-748e-4988-8d29-07ab0e2636e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.467119 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d8b2a-748e-4988-8d29-07ab0e2636e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:20.469022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d8b2a-748e-4988-8d29-07ab0e2636e7 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:20.474720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4a33d-8875-481e-b66b-87095b57eac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e17bb7af1b249d5870c85fd6ff183bc, user_id=ccb0f2ac006245eb87e26a9672952f66, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:20.477447 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4a33d-8875-481e-b66b-87095b57eac5 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:20.492864 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1227/2460] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:20 2022] PUT /identity/v3/projects/6e17bb7af1b249d5870c85fd6ff183bc/users/ccb0f2ac006245eb87e26a9672952f66/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:32:20.499481 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1234/2461] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:20 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:32:20.502784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4a33d-8875-481e-b66b-87095b57eac5 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.503206 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54397c07-df9d-4baf-89f2-6fc4271b3d71 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.503522 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54397c07-df9d-4baf-89f2-6fc4271b3d71 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.503796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54397c07-df9d-4baf-89f2-6fc4271b3d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.533786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-54397c07-df9d-4baf-89f2-6fc4271b3d71 None None] MFA Rules not processed for user `ccb0f2ac006245eb87e26a9672952f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:20.535160 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-54397c07-df9d-4baf-89f2-6fc4271b3d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:20.566666 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4a33d-8875-481e-b66b-87095b57eac5 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:20.604262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1235/2462] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:20 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:20.608920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54397c07-df9d-4baf-89f2-6fc4271b3d71 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.624704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23a9e0-e171-4b6b-9028-eda44251e07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:20.625326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23a9e0-e171-4b6b-9028-eda44251e07e admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.625606 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23a9e0-e171-4b6b-9028-eda44251e07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.626004 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23a9e0-e171-4b6b-9028-eda44251e07e admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.626632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23a9e0-e171-4b6b-9028-eda44251e07e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:20.628672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23a9e0-e171-4b6b-9028-eda44251e07e admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:20.634373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:20.655439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1228/2463] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:20 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:20.656540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23a9e0-e171-4b6b-9028-eda44251e07e admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.661748 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:20.668037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23a9e0-e171-4b6b-9028-eda44251e07e admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:20.673428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59d8e7-6495-4777-b538-9c37880d01ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:20.674044 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59d8e7-6495-4777-b538-9c37880d01ee admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.674317 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59d8e7-6495-4777-b538-9c37880d01ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.674587 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59d8e7-6495-4777-b538-9c37880d01ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.675230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59d8e7-6495-4777-b538-9c37880d01ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:20.677054 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59d8e7-6495-4777-b538-9c37880d01ee admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:20.678617 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54397c07-df9d-4baf-89f2-6fc4271b3d71 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:20.683234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf603684-2f91-4eb1-aa8a-8aaf4e8ba058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:20.683828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf603684-2f91-4eb1-aa8a-8aaf4e8ba058 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.684146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf603684-2f91-4eb1-aa8a-8aaf4e8ba058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.684424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf603684-2f91-4eb1-aa8a-8aaf4e8ba058 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.685076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf603684-2f91-4eb1-aa8a-8aaf4e8ba058 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:20.686971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf603684-2f91-4eb1-aa8a-8aaf4e8ba058 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:20.687258 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1229/2464] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:32:20.691380 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59d8e7-6495-4777-b538-9c37880d01ee admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.701752 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59d8e7-6495-4777-b538-9c37880d01ee admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:20.707454 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663ff2b-fd71-4820-bc0c-ba4e4e0e5b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:20.708167 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663ff2b-fd71-4820-bc0c-ba4e4e0e5b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.708428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663ff2b-fd71-4820-bc0c-ba4e4e0e5b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.708715 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663ff2b-fd71-4820-bc0c-ba4e4e0e5b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.709376 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663ff2b-fd71-4820-bc0c-ba4e4e0e5b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:20.711152 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663ff2b-fd71-4820-bc0c-ba4e4e0e5b57 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:20.716525 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1236/2465] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:20 2022] POST /identity/v3/projects => generated 403 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:20.720620 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf603684-2f91-4eb1-aa8a-8aaf4e8ba058 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.726737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1230/2466] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:32:20.730726 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663ff2b-fd71-4820-bc0c-ba4e4e0e5b57 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.740613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663ff2b-fd71-4820-bc0c-ba4e4e0e5b57 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:20.783687 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf603684-2f91-4eb1-aa8a-8aaf4e8ba058 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:20.788800 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67485c-4944-4a99-8bb1-46c89acf092a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:20.789557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67485c-4944-4a99-8bb1-46c89acf092a admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.789847 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67485c-4944-4a99-8bb1-46c89acf092a admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.790126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67485c-4944-4a99-8bb1-46c89acf092a admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.790761 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67485c-4944-4a99-8bb1-46c89acf092a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:20.792743 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67485c-4944-4a99-8bb1-46c89acf092a admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:20.797778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cfc4f-1772-48e9-9044-7ad4da34c24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:20.798448 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cfc4f-1772-48e9-9044-7ad4da34c24c admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.798732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cfc4f-1772-48e9-9044-7ad4da34c24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.799003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cfc4f-1772-48e9-9044-7ad4da34c24c admin admin] PATH_INFO: `/v3/projects/34950b5c9a734f5cb5820e83dedd4030/users/de73f338e6ef4844979ead86ef2bd16c/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.815475 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cfc4f-1772-48e9-9044-7ad4da34c24c admin admin] RBAC: Authorizing `identity:create_grant(project_id=34950b5c9a734f5cb5820e83dedd4030, user_id=de73f338e6ef4844979ead86ef2bd16c, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:20.816545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1237/2467] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:20 2022] POST /identity/v3/users => generated 308 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:32:20.819744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cfc4f-1772-48e9-9044-7ad4da34c24c admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:20.821062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67485c-4944-4a99-8bb1-46c89acf092a admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.832452 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67485c-4944-4a99-8bb1-46c89acf092a admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:20.835865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1231/2468] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:20 2022] PUT /identity/v3/projects/34950b5c9a734f5cb5820e83dedd4030/users/de73f338e6ef4844979ead86ef2bd16c/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:32:20.841130 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645c23e-43a2-4a41-ac87-0b568b8afa94 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.841413 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645c23e-43a2-4a41-ac87-0b568b8afa94 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.841669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0645c23e-43a2-4a41-ac87-0b568b8afa94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.862347 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-0645c23e-43a2-4a41-ac87-0b568b8afa94 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:20.863623 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0645c23e-43a2-4a41-ac87-0b568b8afa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:20.903596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113be1b-f05d-4b77-8bca-446a0b8e9b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:20.904253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113be1b-f05d-4b77-8bca-446a0b8e9b3e admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.904524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113be1b-f05d-4b77-8bca-446a0b8e9b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.904785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113be1b-f05d-4b77-8bca-446a0b8e9b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.905465 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113be1b-f05d-4b77-8bca-446a0b8e9b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:20.907075 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113be1b-f05d-4b77-8bca-446a0b8e9b3e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:20.917784 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1238/2469] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:32:20.922417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113be1b-f05d-4b77-8bca-446a0b8e9b3e admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.932407 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113be1b-f05d-4b77-8bca-446a0b8e9b3e admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:20.932963 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1232/2470] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:32:20.937591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84c45c-bfa6-4e7a-a7d1-ee1d6922cbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:20.938256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84c45c-bfa6-4e7a-a7d1-ee1d6922cbfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:20.938535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84c45c-bfa6-4e7a-a7d1-ee1d6922cbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:20.938807 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84c45c-bfa6-4e7a-a7d1-ee1d6922cbfd admin admin] PATH_INFO: `/v3/projects/e2366430a38a41948af24fa3747774e3/users/0016237bc3ee40c08d7a441901fe55f4/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:20.942771 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645c23e-43a2-4a41-ac87-0b568b8afa94 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.944259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0645c23e-43a2-4a41-ac87-0b568b8afa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:20.955695 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da84c45c-bfa6-4e7a-a7d1-ee1d6922cbfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2366430a38a41948af24fa3747774e3, user_id=0016237bc3ee40c08d7a441901fe55f4, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:20.958705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da84c45c-bfa6-4e7a-a7d1-ee1d6922cbfd admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:20.973544 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1239/2471] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:20 2022] PUT /identity/v3/projects/e2366430a38a41948af24fa3747774e3/users/0016237bc3ee40c08d7a441901fe55f4/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:32:20.977637 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84c45c-bfa6-4e7a-a7d1-ee1d6922cbfd admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:20.989240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:21.012477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0645c23e-43a2-4a41-ac87-0b568b8afa94 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:21.030676 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:21.050746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84c45c-bfa6-4e7a-a7d1-ee1d6922cbfd admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:21.055516 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82b43f-a08e-4d69-afc4-79124388f934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:21.056180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82b43f-a08e-4d69-afc4-79124388f934 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:21.056461 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82b43f-a08e-4d69-afc4-79124388f934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:21.056731 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82b43f-a08e-4d69-afc4-79124388f934 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:21.057375 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82b43f-a08e-4d69-afc4-79124388f934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:21.058981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82b43f-a08e-4d69-afc4-79124388f934 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:21.069256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1240/2472] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:32:21.070416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1530a-a43e-4512-a81d-c60d966eaf2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:21.071038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1530a-a43e-4512-a81d-c60d966eaf2c service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:21.071309 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1530a-a43e-4512-a81d-c60d966eaf2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:21.071573 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1530a-a43e-4512-a81d-c60d966eaf2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:21.073733 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82b43f-a08e-4d69-afc4-79124388f934 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:21.081870 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1530a-a43e-4512-a81d-c60d966eaf2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:21.083391 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1530a-a43e-4512-a81d-c60d966eaf2c service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:21.083545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82b43f-a08e-4d69-afc4-79124388f934 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:21.088282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d2a8f-4714-4476-ab8f-79c9a22fdb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:21.088942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d2a8f-4714-4476-ab8f-79c9a22fdb0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:21.089224 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d2a8f-4714-4476-ab8f-79c9a22fdb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:21.089495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d2a8f-4714-4476-ab8f-79c9a22fdb0a admin admin] PATH_INFO: `/v3/projects/e2366430a38a41948af24fa3747774e3/users/0016237bc3ee40c08d7a441901fe55f4/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:21.091373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d2a8f-4714-4476-ab8f-79c9a22fdb0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2366430a38a41948af24fa3747774e3, user_id=0016237bc3ee40c08d7a441901fe55f4, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:21.093611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1233/2473] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:21.094028 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d2a8f-4714-4476-ab8f-79c9a22fdb0a admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:21.105529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1241/2474] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:21 2022] PUT /identity/v3/projects/e2366430a38a41948af24fa3747774e3/users/0016237bc3ee40c08d7a441901fe55f4/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:32:21.118046 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4a88c-60bb-4927-b4a1-b7f6b9a38941 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:21.118412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4a88c-60bb-4927-b4a1-b7f6b9a38941 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:21.118766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4a88c-60bb-4927-b4a1-b7f6b9a38941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:21.152774 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-cec4a88c-60bb-4927-b4a1-b7f6b9a38941 None None] MFA Rules not processed for user `0016237bc3ee40c08d7a441901fe55f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:21.154001 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-cec4a88c-60bb-4927-b4a1-b7f6b9a38941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:21.239732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1234/2475] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:21 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:21.251970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d2a8f-4714-4476-ab8f-79c9a22fdb0a admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:21.254786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d52d2a8f-4714-4476-ab8f-79c9a22fdb0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:21.321351 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d2a8f-4714-4476-ab8f-79c9a22fdb0a admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:21.325213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77ba98-b14e-4744-9333-b5e13644b088 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:21.325854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77ba98-b14e-4744-9333-b5e13644b088 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:21.326112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77ba98-b14e-4744-9333-b5e13644b088 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:21.326401 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77ba98-b14e-4744-9333-b5e13644b088 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:21.336987 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77ba98-b14e-4744-9333-b5e13644b088 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:21.338442 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77ba98-b14e-4744-9333-b5e13644b088 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:21.348520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1242/2476] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:21 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:21.520494 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4a88c-60bb-4927-b4a1-b7f6b9a38941 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:21.593852 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4a88c-60bb-4927-b4a1-b7f6b9a38941 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:21.598712 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db5b642-9ba7-4d3b-8c7b-9db2649e8fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:21.599356 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5b642-9ba7-4d3b-8c7b-9db2649e8fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:21.599639 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5b642-9ba7-4d3b-8c7b-9db2649e8fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:21.599909 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db5b642-9ba7-4d3b-8c7b-9db2649e8fef admin admin] PATH_INFO: `/v3/users/ccb0f2ac006245eb87e26a9672952f66` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:21.601091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5b642-9ba7-4d3b-8c7b-9db2649e8fef admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccb0f2ac006245eb87e26a9672952f66)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:21.603040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db5b642-9ba7-4d3b-8c7b-9db2649e8fef admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:21.659004 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4db5b642-9ba7-4d3b-8c7b-9db2649e8fef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccb0f2ac006245eb87e26a9672952f66'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:21.661592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:21.667777 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4db5b642-9ba7-4d3b-8c7b-9db2649e8fef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccb0f2ac006245eb87e26a9672952f66'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:21.681826 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4db5b642-9ba7-4d3b-8c7b-9db2649e8fef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccb0f2ac006245eb87e26a9672952f66'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:21.682561 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4db5b642-9ba7-4d3b-8c7b-9db2649e8fef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccb0f2ac006245eb87e26a9672952f66'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:21.691266 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-4db5b642-9ba7-4d3b-8c7b-9db2649e8fef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccb0f2ac006245eb87e26a9672952f66'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:21.697814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1235/2477] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:21 2022] DELETE /identity/v3/users/ccb0f2ac006245eb87e26a9672952f66 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:21.702410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1488c8-1fa0-4421-9265-2079e6df9970 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:21.702702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1488c8-1fa0-4421-9265-2079e6df9970 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:21.703008 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1488c8-1fa0-4421-9265-2079e6df9970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:21.723318 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-bb1488c8-1fa0-4421-9265-2079e6df9970 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:21.725139 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-bb1488c8-1fa0-4421-9265-2079e6df9970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:21.789871 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1243/2478] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:21.795550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1aec6-e6ab-4139-9ab3-af70c0f87e9d None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:21.795932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1aec6-e6ab-4139-9ab3-af70c0f87e9d None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:21.796330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc1aec6-e6ab-4139-9ab3-af70c0f87e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:21.800729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1488c8-1fa0-4421-9265-2079e6df9970 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:21.802460 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-bb1488c8-1fa0-4421-9265-2079e6df9970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:21.831886 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-0fc1aec6-e6ab-4139-9ab3-af70c0f87e9d None None] MFA Rules not processed for user `de73f338e6ef4844979ead86ef2bd16c`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:21.833868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0fc1aec6-e6ab-4139-9ab3-af70c0f87e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:21.853383 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:21.886013 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1488c8-1fa0-4421-9265-2079e6df9970 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:21.889926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069969e-80f5-4a56-9b51-7b2077ceadc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:21.890690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069969e-80f5-4a56-9b51-7b2077ceadc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:21.891066 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069969e-80f5-4a56-9b51-7b2077ceadc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:21.891438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069969e-80f5-4a56-9b51-7b2077ceadc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:21.901572 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069969e-80f5-4a56-9b51-7b2077ceadc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:21.903093 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c069969e-80f5-4a56-9b51-7b2077ceadc9 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:21.919292 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1236/2479] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:21.919525 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1244/2480] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:32:21.928724 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1aec6-e6ab-4139-9ab3-af70c0f87e9d None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:21.931018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0fc1aec6-e6ab-4139-9ab3-af70c0f87e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:21.995859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc1aec6-e6ab-4139-9ab3-af70c0f87e9d None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:21.999834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d826b-8230-4566-887f-e3c6e1bb4b81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:22.000551 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d826b-8230-4566-887f-e3c6e1bb4b81 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:22.000821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d826b-8230-4566-887f-e3c6e1bb4b81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:22.001086 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d826b-8230-4566-887f-e3c6e1bb4b81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:22.014637 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d826b-8230-4566-887f-e3c6e1bb4b81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:22.016081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d826b-8230-4566-887f-e3c6e1bb4b81 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:22.026878 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1237/2481] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:22.209344 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069969e-80f5-4a56-9b51-7b2077ceadc9 service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:22.211247 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-c069969e-80f5-4a56-9b51-7b2077ceadc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:22.226215 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069969e-80f5-4a56-9b51-7b2077ceadc9 service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:22.232192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d5f67-595a-42d8-bf00-7cb82b8af18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:22.232935 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d5f67-595a-42d8-bf00-7cb82b8af18d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:22.233290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d5f67-595a-42d8-bf00-7cb82b8af18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:22.233623 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d5f67-595a-42d8-bf00-7cb82b8af18d admin admin] PATH_INFO: `/v3/projects/6e17bb7af1b249d5870c85fd6ff183bc` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:22.234987 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d5f67-595a-42d8-bf00-7cb82b8af18d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e17bb7af1b249d5870c85fd6ff183bc)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:22.236939 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d5f67-595a-42d8-bf00-7cb82b8af18d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:22.291535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-094d5f67-595a-42d8-bf00-7cb82b8af18d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e17bb7af1b249d5870c85fd6ff183bc', 'request_id': 'req-094d5f67-595a-42d8-bf00-7cb82b8af18d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:22.298651 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-094d5f67-595a-42d8-bf00-7cb82b8af18d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e17bb7af1b249d5870c85fd6ff183bc', 'request_id': 'req-094d5f67-595a-42d8-bf00-7cb82b8af18d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:22.308157 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-094d5f67-595a-42d8-bf00-7cb82b8af18d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e17bb7af1b249d5870c85fd6ff183bc', 'request_id': 'req-094d5f67-595a-42d8-bf00-7cb82b8af18d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:22.324698 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-094d5f67-595a-42d8-bf00-7cb82b8af18d admin admin] The token cache is being invalidate because project 6e17bb7af1b249d5870c85fd6ff183bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:22.325070 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-094d5f67-595a-42d8-bf00-7cb82b8af18d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:22.325790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-094d5f67-595a-42d8-bf00-7cb82b8af18d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:22.338990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1245/2482] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:22 2022] DELETE /identity/v3/projects/6e17bb7af1b249d5870c85fd6ff183bc => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:22.339543 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d826b-8230-4566-887f-e3c6e1bb4b81 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:22.343257 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d826b-8230-4566-887f-e3c6e1bb4b81 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:22.398866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1bbab-767a-41cf-90fb-24bdda9fd475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:22.399502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1bbab-767a-41cf-90fb-24bdda9fd475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:22.399778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1bbab-767a-41cf-90fb-24bdda9fd475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:22.400074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1bbab-767a-41cf-90fb-24bdda9fd475 admin admin] PATH_INFO: `/v3/users/0016237bc3ee40c08d7a441901fe55f4` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:22.401185 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1bbab-767a-41cf-90fb-24bdda9fd475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0016237bc3ee40c08d7a441901fe55f4)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:22.403111 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1bbab-767a-41cf-90fb-24bdda9fd475 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:22.404590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:22.455369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-aaa1bbab-767a-41cf-90fb-24bdda9fd475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0016237bc3ee40c08d7a441901fe55f4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:22.462414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-aaa1bbab-767a-41cf-90fb-24bdda9fd475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0016237bc3ee40c08d7a441901fe55f4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:22.474896 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-aaa1bbab-767a-41cf-90fb-24bdda9fd475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0016237bc3ee40c08d7a441901fe55f4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:22.475641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-aaa1bbab-767a-41cf-90fb-24bdda9fd475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0016237bc3ee40c08d7a441901fe55f4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:22.483887 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-aaa1bbab-767a-41cf-90fb-24bdda9fd475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0016237bc3ee40c08d7a441901fe55f4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:22.490696 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1238/2483] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:22 2022] DELETE /identity/v3/users/0016237bc3ee40c08d7a441901fe55f4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:22.790835 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d5f67-595a-42d8-bf00-7cb82b8af18d admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:22.792519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-094d5f67-595a-42d8-bf00-7cb82b8af18d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:22.807472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d5f67-595a-42d8-bf00-7cb82b8af18d admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:22.868086 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c40549-3639-4e40-9c0e-479450e5e050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:22.868633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c40549-3639-4e40-9c0e-479450e5e050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:22.868917 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c40549-3639-4e40-9c0e-479450e5e050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:22.869197 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c40549-3639-4e40-9c0e-479450e5e050 admin admin] PATH_INFO: `/v3/projects/e2366430a38a41948af24fa3747774e3` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:22.870392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c40549-3639-4e40-9c0e-479450e5e050 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2366430a38a41948af24fa3747774e3)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:22.872411 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c40549-3639-4e40-9c0e-479450e5e050 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:22.926592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-49c40549-3639-4e40-9c0e-479450e5e050 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2366430a38a41948af24fa3747774e3', 'request_id': 'req-49c40549-3639-4e40-9c0e-479450e5e050'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:22.933556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-49c40549-3639-4e40-9c0e-479450e5e050 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2366430a38a41948af24fa3747774e3', 'request_id': 'req-49c40549-3639-4e40-9c0e-479450e5e050'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:22.944248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-49c40549-3639-4e40-9c0e-479450e5e050 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2366430a38a41948af24fa3747774e3', 'request_id': 'req-49c40549-3639-4e40-9c0e-479450e5e050'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:22.946613 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:22.957369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-49c40549-3639-4e40-9c0e-479450e5e050 admin admin] The token cache is being invalidate because project e2366430a38a41948af24fa3747774e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:22.957887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-49c40549-3639-4e40-9c0e-479450e5e050 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:22.958560 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-49c40549-3639-4e40-9c0e-479450e5e050 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:22.960092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1246/2484] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:22 2022] DELETE /identity/v3/projects/e2366430a38a41948af24fa3747774e3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:22.965094 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c54b3e-27f5-4126-8ad7-0929f0716f50 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:22.965452 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c54b3e-27f5-4126-8ad7-0929f0716f50 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:22.965693 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c54b3e-27f5-4126-8ad7-0929f0716f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:22.986599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-c8c54b3e-27f5-4126-8ad7-0929f0716f50 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:22.987820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c8c54b3e-27f5-4126-8ad7-0929f0716f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:23.052212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1239/2485] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:23.059960 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a8280-36d9-41e4-8b42-aedae11403fd None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:23.060276 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a8280-36d9-41e4-8b42-aedae11403fd None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:23.060545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a8280-36d9-41e4-8b42-aedae11403fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:23.080562 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-ea9a8280-36d9-41e4-8b42-aedae11403fd None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:23.081819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ea9a8280-36d9-41e4-8b42-aedae11403fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:23.090862 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1247/2486] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:23.095185 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c54b3e-27f5-4126-8ad7-0929f0716f50 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:23.105300 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c54b3e-27f5-4126-8ad7-0929f0716f50 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:23.110505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fba15-7f3f-4bd2-8e28-46ddbe56c0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:23.111101 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fba15-7f3f-4bd2-8e28-46ddbe56c0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:23.111365 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fba15-7f3f-4bd2-8e28-46ddbe56c0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:23.111648 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fba15-7f3f-4bd2-8e28-46ddbe56c0d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:23.112343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372fba15-7f3f-4bd2-8e28-46ddbe56c0d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:32:23.112643 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372fba15-7f3f-4bd2-8e28-46ddbe56c0d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:23.114302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372fba15-7f3f-4bd2-8e28-46ddbe56c0d0 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:23.126239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1240/2487] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:32:23.130901 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a8280-36d9-41e4-8b42-aedae11403fd None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:23.142842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a8280-36d9-41e4-8b42-aedae11403fd None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:23.151177 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46a866-e082-43c1-b71b-10649be57606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:23.151985 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46a866-e082-43c1-b71b-10649be57606 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:23.152378 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46a866-e082-43c1-b71b-10649be57606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:23.152667 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46a866-e082-43c1-b71b-10649be57606 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:23.153558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46a866-e082-43c1-b71b-10649be57606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:23.156441 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46a866-e082-43c1-b71b-10649be57606 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:23.190858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1248/2488] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:23 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:23.195233 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fba15-7f3f-4bd2-8e28-46ddbe56c0d0 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:23.205835 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fba15-7f3f-4bd2-8e28-46ddbe56c0d0 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:23.271796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37aa7d-aefb-4dea-a174-ee4afcbec727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:23.272478 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37aa7d-aefb-4dea-a174-ee4afcbec727 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:23.272738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37aa7d-aefb-4dea-a174-ee4afcbec727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:23.273652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37aa7d-aefb-4dea-a174-ee4afcbec727 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:23.274294 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f37aa7d-aefb-4dea-a174-ee4afcbec727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:23.276982 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f37aa7d-aefb-4dea-a174-ee4afcbec727 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:23.289578 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:23.312258 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1241/2489] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:23 2022] POST /identity/v3/users => generated 284 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:23.313722 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46a866-e082-43c1-b71b-10649be57606 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:23.324968 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46a866-e082-43c1-b71b-10649be57606 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:23.330073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c6473-791d-4060-96c0-7528dd105217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:23.330688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c6473-791d-4060-96c0-7528dd105217 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:23.330970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c6473-791d-4060-96c0-7528dd105217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:23.331244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c6473-791d-4060-96c0-7528dd105217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:23.331931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c6473-791d-4060-96c0-7528dd105217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:23.333599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c6473-791d-4060-96c0-7528dd105217 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:23.344348 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1249/2490] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:23.348345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37aa7d-aefb-4dea-a174-ee4afcbec727 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:23.358806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37aa7d-aefb-4dea-a174-ee4afcbec727 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:23.364018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24397b-5b30-419f-a852-17721fd883d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:23.364638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24397b-5b30-419f-a852-17721fd883d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:23.364902 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24397b-5b30-419f-a852-17721fd883d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:23.365183 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24397b-5b30-419f-a852-17721fd883d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:23.365822 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24397b-5b30-419f-a852-17721fd883d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:23.367403 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24397b-5b30-419f-a852-17721fd883d2 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:23.378946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1242/2491] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:23.383052 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c6473-791d-4060-96c0-7528dd105217 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:23.393185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c6473-791d-4060-96c0-7528dd105217 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:23.397991 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd88ff-5eab-4e5f-98da-8afeb378c28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:23.398658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd88ff-5eab-4e5f-98da-8afeb378c28f admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:23.398940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd88ff-5eab-4e5f-98da-8afeb378c28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:23.399222 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd88ff-5eab-4e5f-98da-8afeb378c28f admin admin] PATH_INFO: `/v3/projects/e426d64fbfaf402a9bc3339322854664/users/fcf106fc439a4603ac9b520ba960c5ed/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:23.401497 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fe056-3327-4792-a691-afb9658e2b74 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:23.401776 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fe056-3327-4792-a691-afb9658e2b74 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:23.402040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fe056-3327-4792-a691-afb9658e2b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:23.415758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd88ff-5eab-4e5f-98da-8afeb378c28f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e426d64fbfaf402a9bc3339322854664, user_id=fcf106fc439a4603ac9b520ba960c5ed, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:23.418554 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd88ff-5eab-4e5f-98da-8afeb378c28f admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:23.421624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-e91fe056-3327-4792-a691-afb9658e2b74 None None] MFA Rules not processed for user `de73f338e6ef4844979ead86ef2bd16c`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:23.422830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e91fe056-3327-4792-a691-afb9658e2b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:23.435323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1250/2492] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:23 2022] PUT /identity/v3/projects/e426d64fbfaf402a9bc3339322854664/users/fcf106fc439a4603ac9b520ba960c5ed/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:23.445902 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490de757-4678-4972-8cda-8f34bf3b222c None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:23.446182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490de757-4678-4972-8cda-8f34bf3b222c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:23.446457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490de757-4678-4972-8cda-8f34bf3b222c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:23.475057 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-490de757-4678-4972-8cda-8f34bf3b222c None None] MFA Rules not processed for user `fcf106fc439a4603ac9b520ba960c5ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:23.476308 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-490de757-4678-4972-8cda-8f34bf3b222c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:23.484806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1243/2493] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:23.551550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1251/2494] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:32:23.556285 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fe056-3327-4792-a691-afb9658e2b74 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:23.567634 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fe056-3327-4792-a691-afb9658e2b74 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:23.626917 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a77250-5fd4-4f61-8abf-1e0efc135341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:23.627580 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a77250-5fd4-4f61-8abf-1e0efc135341 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:23.627859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a77250-5fd4-4f61-8abf-1e0efc135341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:23.628648 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a77250-5fd4-4f61-8abf-1e0efc135341 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:23.628888 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a77250-5fd4-4f61-8abf-1e0efc135341 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:23.631079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a77250-5fd4-4f61-8abf-1e0efc135341 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:23.661089 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1244/2495] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:23 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:32:23.669131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490de757-4678-4972-8cda-8f34bf3b222c None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:23.681270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490de757-4678-4972-8cda-8f34bf3b222c None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:23.742039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb2232-15ff-4ded-bf3b-e838bb1b918d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:23.742816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb2232-15ff-4ded-bf3b-e838bb1b918d admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:23.742944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb2232-15ff-4ded-bf3b-e838bb1b918d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:23.743207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb2232-15ff-4ded-bf3b-e838bb1b918d admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:23.744018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb2232-15ff-4ded-bf3b-e838bb1b918d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:23.745909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb2232-15ff-4ded-bf3b-e838bb1b918d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:23.768502 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1252/2496] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:23 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:23.772801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a77250-5fd4-4f61-8abf-1e0efc135341 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:23.783462 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a77250-5fd4-4f61-8abf-1e0efc135341 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:23.788176 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c313f6e-d8e2-4580-94ff-8ad197e1b1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:23.788791 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c313f6e-d8e2-4580-94ff-8ad197e1b1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:23.789083 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c313f6e-d8e2-4580-94ff-8ad197e1b1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:23.789342 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c313f6e-d8e2-4580-94ff-8ad197e1b1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:23.789984 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c313f6e-d8e2-4580-94ff-8ad197e1b1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:23.791655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c313f6e-d8e2-4580-94ff-8ad197e1b1a9 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:23.803418 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1245/2497] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:23.808086 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb2232-15ff-4ded-bf3b-e838bb1b918d admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:23.820033 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb2232-15ff-4ded-bf3b-e838bb1b918d admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:23.824804 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1ec64-367e-421f-8894-4c61044d0a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:23.825472 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1ec64-367e-421f-8894-4c61044d0a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:23.825753 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1ec64-367e-421f-8894-4c61044d0a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:23.826032 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1ec64-367e-421f-8894-4c61044d0a96 admin admin] PATH_INFO: `/v3/projects/f5c5e6a3cbf647ea9728ea69b539bdfc/users/7c1ab7a914cd43f5a0a3def22707d9ae/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:23.828416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=94,keystone:DELETE=1,keystone:INSERT=5 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:23.844236 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc1ec64-367e-421f-8894-4c61044d0a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5c5e6a3cbf647ea9728ea69b539bdfc, user_id=7c1ab7a914cd43f5a0a3def22707d9ae, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:23.846930 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc1ec64-367e-421f-8894-4c61044d0a96 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:23.861196 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1253/2498] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:23 2022] PUT /identity/v3/projects/f5c5e6a3cbf647ea9728ea69b539bdfc/users/7c1ab7a914cd43f5a0a3def22707d9ae/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:32:23.865430 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c313f6e-d8e2-4580-94ff-8ad197e1b1a9 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:23.875302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c313f6e-d8e2-4580-94ff-8ad197e1b1a9 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:23.936784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87da1c-5bdf-44cd-8b1d-733a65556fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:23.937407 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87da1c-5bdf-44cd-8b1d-733a65556fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:23.937680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87da1c-5bdf-44cd-8b1d-733a65556fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:23.937945 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87da1c-5bdf-44cd-8b1d-733a65556fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:23.938585 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87da1c-5bdf-44cd-8b1d-733a65556fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:23.940243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87da1c-5bdf-44cd-8b1d-733a65556fe6 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:23.946179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:23.952040 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1246/2499] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:23.956292 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1ec64-367e-421f-8894-4c61044d0a96 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:23.967696 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1ec64-367e-421f-8894-4c61044d0a96 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:23.973369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce973c0a-2511-4e0d-b055-bee50210180f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:23.973850 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce973c0a-2511-4e0d-b055-bee50210180f admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:23.974064 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce973c0a-2511-4e0d-b055-bee50210180f admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:23.974383 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce973c0a-2511-4e0d-b055-bee50210180f admin admin] PATH_INFO: `/v3/projects/f5c5e6a3cbf647ea9728ea69b539bdfc/users/7c1ab7a914cd43f5a0a3def22707d9ae/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:23.976583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce973c0a-2511-4e0d-b055-bee50210180f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5c5e6a3cbf647ea9728ea69b539bdfc, user_id=7c1ab7a914cd43f5a0a3def22707d9ae, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:23.979229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce973c0a-2511-4e0d-b055-bee50210180f admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:23.991012 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1254/2500] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:23 2022] PUT /identity/v3/projects/f5c5e6a3cbf647ea9728ea69b539bdfc/users/7c1ab7a914cd43f5a0a3def22707d9ae/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:32:24.003382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5b0aa-748d-4cb6-a42f-6795e5763777 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:24.003728 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5b0aa-748d-4cb6-a42f-6795e5763777 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:24.004217 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e5b0aa-748d-4cb6-a42f-6795e5763777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:24.034469 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-60e5b0aa-748d-4cb6-a42f-6795e5763777 None None] MFA Rules not processed for user `7c1ab7a914cd43f5a0a3def22707d9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:24.035953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-60e5b0aa-748d-4cb6-a42f-6795e5763777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:24.123100 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1247/2501] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:24.139578 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce973c0a-2511-4e0d-b055-bee50210180f admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:24.141729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ce973c0a-2511-4e0d-b055-bee50210180f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:24.205151 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce973c0a-2511-4e0d-b055-bee50210180f admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:24.209072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab2d15-0a18-49b3-a2c2-8a808f8d9bd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:24.209689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab2d15-0a18-49b3-a2c2-8a808f8d9bd2 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:24.209979 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab2d15-0a18-49b3-a2c2-8a808f8d9bd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:24.210269 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab2d15-0a18-49b3-a2c2-8a808f8d9bd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:24.219424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab2d15-0a18-49b3-a2c2-8a808f8d9bd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:24.220899 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab2d15-0a18-49b3-a2c2-8a808f8d9bd2 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:24.231920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1255/2502] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:24.319745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5b0aa-748d-4cb6-a42f-6795e5763777 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:24.331379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e5b0aa-748d-4cb6-a42f-6795e5763777 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:24.335433 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5cdade-b0d0-421b-a98b-d2e54a63a40d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:24.336086 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cdade-b0d0-421b-a98b-d2e54a63a40d service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:24.336343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cdade-b0d0-421b-a98b-d2e54a63a40d service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:24.336721 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5cdade-b0d0-421b-a98b-d2e54a63a40d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:24.396665 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5cdade-b0d0-421b-a98b-d2e54a63a40d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:24.398166 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5cdade-b0d0-421b-a98b-d2e54a63a40d service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:24.407563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1248/2503] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:29.230933 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab2d15-0a18-49b3-a2c2-8a808f8d9bd2 service cinder] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:29.303439 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab2d15-0a18-49b3-a2c2-8a808f8d9bd2 service cinder] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:29.309584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43656e-21c3-4b2a-9f3b-0e9ea88de850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:29.310203 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43656e-21c3-4b2a-9f3b-0e9ea88de850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:29.310485 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43656e-21c3-4b2a-9f3b-0e9ea88de850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:29.310756 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43656e-21c3-4b2a-9f3b-0e9ea88de850 admin admin] PATH_INFO: `/v3/users/fcf106fc439a4603ac9b520ba960c5ed` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:29.311900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43656e-21c3-4b2a-9f3b-0e9ea88de850 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcf106fc439a4603ac9b520ba960c5ed)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:29.313993 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43656e-21c3-4b2a-9f3b-0e9ea88de850 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:29.365697 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-6d43656e-21c3-4b2a-9f3b-0e9ea88de850 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcf106fc439a4603ac9b520ba960c5ed'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:29.379064 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-6d43656e-21c3-4b2a-9f3b-0e9ea88de850 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcf106fc439a4603ac9b520ba960c5ed'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:29.385898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-6d43656e-21c3-4b2a-9f3b-0e9ea88de850 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcf106fc439a4603ac9b520ba960c5ed'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:29.386590 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-6d43656e-21c3-4b2a-9f3b-0e9ea88de850 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcf106fc439a4603ac9b520ba960c5ed'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:29.387193 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-6d43656e-21c3-4b2a-9f3b-0e9ea88de850 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcf106fc439a4603ac9b520ba960c5ed'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:29.388866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:29.411170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1256/2504] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:29 2022] DELETE /identity/v3/users/fcf106fc439a4603ac9b520ba960c5ed => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:29.416037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdaea7-280e-491b-9a6a-a99060d336aa None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:29.416705 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdaea7-280e-491b-9a6a-a99060d336aa None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:29.416705 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdaea7-280e-491b-9a6a-a99060d336aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:29.447717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-4dfdaea7-280e-491b-9a6a-a99060d336aa None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:29.448993 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4dfdaea7-280e-491b-9a6a-a99060d336aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:29.524298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1249/2505] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:29.534541 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43656e-21c3-4b2a-9f3b-0e9ea88de850 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:29.536127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6d43656e-21c3-4b2a-9f3b-0e9ea88de850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:29.599388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43656e-21c3-4b2a-9f3b-0e9ea88de850 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:29.603580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d12657-52b3-4816-a48d-949bc69999ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:29.604238 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d12657-52b3-4816-a48d-949bc69999ad service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:29.604521 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d12657-52b3-4816-a48d-949bc69999ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:29.604796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d12657-52b3-4816-a48d-949bc69999ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:29.615779 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d12657-52b3-4816-a48d-949bc69999ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:29.617249 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d12657-52b3-4816-a48d-949bc69999ad service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:29.627603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1257/2506] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:29.939628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdaea7-280e-491b-9a6a-a99060d336aa None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:29.941198 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4dfdaea7-280e-491b-9a6a-a99060d336aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:29.953973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdaea7-280e-491b-9a6a-a99060d336aa None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:29.959406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:29.960260 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:29.960628 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:29.960990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2 admin admin] PATH_INFO: `/v3/projects/e426d64fbfaf402a9bc3339322854664` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:29.962195 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e426d64fbfaf402a9bc3339322854664)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:29.964206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:29.992373 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:30.027593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e426d64fbfaf402a9bc3339322854664', 'request_id': 'req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:30.036190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e426d64fbfaf402a9bc3339322854664', 'request_id': 'req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:30.043143 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e426d64fbfaf402a9bc3339322854664', 'request_id': 'req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:30.056866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2 admin admin] The token cache is being invalidate because project e426d64fbfaf402a9bc3339322854664 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:30.057253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:30.057904 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:30.059407 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1250/2507] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:29 2022] DELETE /identity/v3/projects/e426d64fbfaf402a9bc3339322854664 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:30.063650 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d12657-52b3-4816-a48d-949bc69999ad service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:30.074603 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d12657-52b3-4816-a48d-949bc69999ad service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:30.132954 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2932e8f2-f429-48d1-aaa2-8158437fdc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:30.133514 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932e8f2-f429-48d1-aaa2-8158437fdc13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:30.134439 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932e8f2-f429-48d1-aaa2-8158437fdc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:30.134439 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2932e8f2-f429-48d1-aaa2-8158437fdc13 admin admin] PATH_INFO: `/v3/users/7c1ab7a914cd43f5a0a3def22707d9ae` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:30.135393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932e8f2-f429-48d1-aaa2-8158437fdc13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c1ab7a914cd43f5a0a3def22707d9ae)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:30.137419 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2932e8f2-f429-48d1-aaa2-8158437fdc13 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:30.189288 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-2932e8f2-f429-48d1-aaa2-8158437fdc13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c1ab7a914cd43f5a0a3def22707d9ae'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:30.202824 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-2932e8f2-f429-48d1-aaa2-8158437fdc13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c1ab7a914cd43f5a0a3def22707d9ae'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:30.210101 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-2932e8f2-f429-48d1-aaa2-8158437fdc13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c1ab7a914cd43f5a0a3def22707d9ae'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:30.210846 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-2932e8f2-f429-48d1-aaa2-8158437fdc13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c1ab7a914cd43f5a0a3def22707d9ae'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:30.211595 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-2932e8f2-f429-48d1-aaa2-8158437fdc13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c1ab7a914cd43f5a0a3def22707d9ae'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:30.226851 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1258/2508] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:30 2022] DELETE /identity/v3/users/7c1ab7a914cd43f5a0a3def22707d9ae => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:30.548064 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:30.550115 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:30.571848 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa7fc5-4de6-4ecd-9237-b3a84ce575b2 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:30.633164 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34961000-0c0c-4986-a270-239e388dfe7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:30.633374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34961000-0c0c-4986-a270-239e388dfe7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:30.633908 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34961000-0c0c-4986-a270-239e388dfe7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:30.634088 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34961000-0c0c-4986-a270-239e388dfe7c admin admin] PATH_INFO: `/v3/projects/f5c5e6a3cbf647ea9728ea69b539bdfc` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:30.635382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34961000-0c0c-4986-a270-239e388dfe7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5c5e6a3cbf647ea9728ea69b539bdfc)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:30.637427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34961000-0c0c-4986-a270-239e388dfe7c admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:30.687451 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-34961000-0c0c-4986-a270-239e388dfe7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5c5e6a3cbf647ea9728ea69b539bdfc', 'request_id': 'req-34961000-0c0c-4986-a270-239e388dfe7c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:30.693282 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-34961000-0c0c-4986-a270-239e388dfe7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5c5e6a3cbf647ea9728ea69b539bdfc', 'request_id': 'req-34961000-0c0c-4986-a270-239e388dfe7c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:30.699530 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-34961000-0c0c-4986-a270-239e388dfe7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5c5e6a3cbf647ea9728ea69b539bdfc', 'request_id': 'req-34961000-0c0c-4986-a270-239e388dfe7c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:30.715539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-34961000-0c0c-4986-a270-239e388dfe7c admin admin] The token cache is being invalidate because project f5c5e6a3cbf647ea9728ea69b539bdfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:30.715766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-34961000-0c0c-4986-a270-239e388dfe7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:30.716470 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-34961000-0c0c-4986-a270-239e388dfe7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:30.718009 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1251/2509] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:30 2022] DELETE /identity/v3/projects/f5c5e6a3cbf647ea9728ea69b539bdfc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:30.723122 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029548ab-81cf-4fe1-8498-5ca060fe16ce None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:30.723653 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029548ab-81cf-4fe1-8498-5ca060fe16ce None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:30.723653 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029548ab-81cf-4fe1-8498-5ca060fe16ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:30.746288 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-029548ab-81cf-4fe1-8498-5ca060fe16ce None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:30.748060 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-029548ab-81cf-4fe1-8498-5ca060fe16ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:30.759751 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:30.829709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1259/2510] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:30.834081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8f6d0-2915-4f68-acc0-abeeebe58d6d None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:30.834458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8f6d0-2915-4f68-acc0-abeeebe58d6d None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:30.834752 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8f6d0-2915-4f68-acc0-abeeebe58d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:30.856139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-78e8f6d0-2915-4f68-acc0-abeeebe58d6d None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:30.857373 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-78e8f6d0-2915-4f68-acc0-abeeebe58d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:30.865350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1252/2511] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:30.869663 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029548ab-81cf-4fe1-8498-5ca060fe16ce None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:30.879436 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029548ab-81cf-4fe1-8498-5ca060fe16ce None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:30.884311 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858072f9-030b-4962-b9c3-98486ef03e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:30.884932 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858072f9-030b-4962-b9c3-98486ef03e5c admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:30.885216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858072f9-030b-4962-b9c3-98486ef03e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:30.885488 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858072f9-030b-4962-b9c3-98486ef03e5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:30.886172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858072f9-030b-4962-b9c3-98486ef03e5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:32:30.886469 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858072f9-030b-4962-b9c3-98486ef03e5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:30.888130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858072f9-030b-4962-b9c3-98486ef03e5c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:30.901060 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1260/2512] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:32:30.905971 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8f6d0-2915-4f68-acc0-abeeebe58d6d None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:30.915504 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8f6d0-2915-4f68-acc0-abeeebe58d6d None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:30.920580 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903abdf4-1db9-48b0-bf1e-480961500087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:30.921197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903abdf4-1db9-48b0-bf1e-480961500087 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:30.921473 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903abdf4-1db9-48b0-bf1e-480961500087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:30.921745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903abdf4-1db9-48b0-bf1e-480961500087 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:30.922369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903abdf4-1db9-48b0-bf1e-480961500087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:30.924361 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903abdf4-1db9-48b0-bf1e-480961500087 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:30.953863 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1253/2513] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:30 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:32:30.958025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858072f9-030b-4962-b9c3-98486ef03e5c admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:30.968416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858072f9-030b-4962-b9c3-98486ef03e5c admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:31.030988 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97290ccb-9e4a-474d-8fa7-e1ddd5476cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:31.031609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97290ccb-9e4a-474d-8fa7-e1ddd5476cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:31.031927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97290ccb-9e4a-474d-8fa7-e1ddd5476cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:31.032245 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97290ccb-9e4a-474d-8fa7-e1ddd5476cc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:31.032873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97290ccb-9e4a-474d-8fa7-e1ddd5476cc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:31.034820 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97290ccb-9e4a-474d-8fa7-e1ddd5476cc3 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:31.060647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1261/2514] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:30 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:31.061445 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903abdf4-1db9-48b0-bf1e-480961500087 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:31.075862 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903abdf4-1db9-48b0-bf1e-480961500087 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:31.082017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94bf1c-708d-4453-a410-98541bfeadbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:31.082550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94bf1c-708d-4453-a410-98541bfeadbf admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:31.082818 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94bf1c-708d-4453-a410-98541bfeadbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:31.083085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94bf1c-708d-4453-a410-98541bfeadbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:31.083717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94bf1c-708d-4453-a410-98541bfeadbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:31.085402 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94bf1c-708d-4453-a410-98541bfeadbf admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:31.087853 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:31.102928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97290ccb-9e4a-474d-8fa7-e1ddd5476cc3 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:31.108441 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1254/2515] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:32:31.121319 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97290ccb-9e4a-474d-8fa7-e1ddd5476cc3 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:31.128758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360847d3-511e-4fc5-be1f-1d38c5b274a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:31.129806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360847d3-511e-4fc5-be1f-1d38c5b274a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:31.130285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360847d3-511e-4fc5-be1f-1d38c5b274a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:31.130752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360847d3-511e-4fc5-be1f-1d38c5b274a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:31.131858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360847d3-511e-4fc5-be1f-1d38c5b274a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:31.134217 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360847d3-511e-4fc5-be1f-1d38c5b274a0 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:31.147304 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1262/2516] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:31.151527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94bf1c-708d-4453-a410-98541bfeadbf admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:31.162452 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94bf1c-708d-4453-a410-98541bfeadbf admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:31.168064 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb7020-8b1a-4700-b715-c83b1f9dcef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:31.168739 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb7020-8b1a-4700-b715-c83b1f9dcef2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:31.169018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb7020-8b1a-4700-b715-c83b1f9dcef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:31.169288 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb7020-8b1a-4700-b715-c83b1f9dcef2 admin admin] PATH_INFO: `/v3/projects/f855ee64a5bb488fb18c74551fc6ca05/users/8ed6a5980077441fb60aef738726663b/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:31.185851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb7020-8b1a-4700-b715-c83b1f9dcef2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f855ee64a5bb488fb18c74551fc6ca05, user_id=8ed6a5980077441fb60aef738726663b, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:31.188616 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb7020-8b1a-4700-b715-c83b1f9dcef2 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:31.204031 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1255/2517] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:31 2022] PUT /identity/v3/projects/f855ee64a5bb488fb18c74551fc6ca05/users/8ed6a5980077441fb60aef738726663b/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:31.216094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72af8e4-d335-42bb-9568-5ddddb8a01b2 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:31.216383 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72af8e4-d335-42bb-9568-5ddddb8a01b2 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:31.216651 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72af8e4-d335-42bb-9568-5ddddb8a01b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:31.245392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-b72af8e4-d335-42bb-9568-5ddddb8a01b2 None None] MFA Rules not processed for user `8ed6a5980077441fb60aef738726663b`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:31.246620 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b72af8e4-d335-42bb-9568-5ddddb8a01b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:31.317973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1263/2518] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:31.323158 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb7020-8b1a-4700-b715-c83b1f9dcef2 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:31.395845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb7020-8b1a-4700-b715-c83b1f9dcef2 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:31.401003 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62459201-b775-4591-9532-0b0baac7461b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:31.401556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62459201-b775-4591-9532-0b0baac7461b admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:31.401827 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62459201-b775-4591-9532-0b0baac7461b admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:31.402089 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62459201-b775-4591-9532-0b0baac7461b admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:31.402700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62459201-b775-4591-9532-0b0baac7461b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:31.404646 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62459201-b775-4591-9532-0b0baac7461b admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:31.433912 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1256/2519] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:31 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:31.439655 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72af8e4-d335-42bb-9568-5ddddb8a01b2 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:31.451062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72af8e4-d335-42bb-9568-5ddddb8a01b2 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:31.456066 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:31.510776 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada82d3-bf33-49b1-8566-77c84bf79a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:31.511393 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada82d3-bf33-49b1-8566-77c84bf79a4c admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:31.511670 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada82d3-bf33-49b1-8566-77c84bf79a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:31.511938 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ada82d3-bf33-49b1-8566-77c84bf79a4c admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:31.512599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada82d3-bf33-49b1-8566-77c84bf79a4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:31.514505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ada82d3-bf33-49b1-8566-77c84bf79a4c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:31.538073 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1264/2520] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:31 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:31.542892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62459201-b775-4591-9532-0b0baac7461b admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:31.553842 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62459201-b775-4591-9532-0b0baac7461b admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:31.558512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9715f67-5e55-410c-8ba4-408fa8008832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:31.559158 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9715f67-5e55-410c-8ba4-408fa8008832 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:31.559391 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9715f67-5e55-410c-8ba4-408fa8008832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:31.559681 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9715f67-5e55-410c-8ba4-408fa8008832 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:31.560353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9715f67-5e55-410c-8ba4-408fa8008832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:31.561965 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9715f67-5e55-410c-8ba4-408fa8008832 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:31.573118 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1257/2521] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:31.577479 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada82d3-bf33-49b1-8566-77c84bf79a4c admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:31.588500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ada82d3-bf33-49b1-8566-77c84bf79a4c admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:31.593582 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cc260-7184-4264-b1ec-23589c644e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:31.594226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cc260-7184-4264-b1ec-23589c644e9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:31.594502 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cc260-7184-4264-b1ec-23589c644e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:31.594764 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cc260-7184-4264-b1ec-23589c644e9e admin admin] PATH_INFO: `/v3/projects/e6be2831107547dfb8c339a761ef6fac/users/95be2aeac83a4e818df54809ece3099c/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:31.609940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715cc260-7184-4264-b1ec-23589c644e9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6be2831107547dfb8c339a761ef6fac, user_id=95be2aeac83a4e818df54809ece3099c, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:31.612619 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715cc260-7184-4264-b1ec-23589c644e9e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:31.628010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1265/2522] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:31 2022] PUT /identity/v3/projects/e6be2831107547dfb8c339a761ef6fac/users/95be2aeac83a4e818df54809ece3099c/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:31.632237 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9715f67-5e55-410c-8ba4-408fa8008832 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:31.642525 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9715f67-5e55-410c-8ba4-408fa8008832 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:31.701110 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a385fd-5f62-4de3-b6ec-75d2ed61ff07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:31.701751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a385fd-5f62-4de3-b6ec-75d2ed61ff07 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:31.702049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a385fd-5f62-4de3-b6ec-75d2ed61ff07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:31.702347 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a385fd-5f62-4de3-b6ec-75d2ed61ff07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:31.702960 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a385fd-5f62-4de3-b6ec-75d2ed61ff07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:31.704623 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a385fd-5f62-4de3-b6ec-75d2ed61ff07 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:31.714960 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1258/2523] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:31.719308 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cc260-7184-4264-b1ec-23589c644e9e admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:31.730357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cc260-7184-4264-b1ec-23589c644e9e admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:31.735464 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00d82f-f1d2-4b1e-b8b5-3da288ddd2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:31.736272 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00d82f-f1d2-4b1e-b8b5-3da288ddd2b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:31.736604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00d82f-f1d2-4b1e-b8b5-3da288ddd2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:31.736940 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00d82f-f1d2-4b1e-b8b5-3da288ddd2b9 admin admin] PATH_INFO: `/v3/projects/e6be2831107547dfb8c339a761ef6fac/users/95be2aeac83a4e818df54809ece3099c/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:31.739021 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00d82f-f1d2-4b1e-b8b5-3da288ddd2b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6be2831107547dfb8c339a761ef6fac, user_id=95be2aeac83a4e818df54809ece3099c, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:31.741757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00d82f-f1d2-4b1e-b8b5-3da288ddd2b9 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:31.758127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1266/2524] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:31 2022] PUT /identity/v3/projects/e6be2831107547dfb8c339a761ef6fac/users/95be2aeac83a4e818df54809ece3099c/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:31.763617 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ad9f7-15af-4613-8a3c-48463a946562 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:31.763903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ad9f7-15af-4613-8a3c-48463a946562 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:31.764227 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ad9f7-15af-4613-8a3c-48463a946562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:31.793425 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-659ad9f7-15af-4613-8a3c-48463a946562 None None] MFA Rules not processed for user `95be2aeac83a4e818df54809ece3099c`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:31.794664 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-659ad9f7-15af-4613-8a3c-48463a946562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:31.829733 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:31.866968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1259/2525] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:31.877715 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00d82f-f1d2-4b1e-b8b5-3da288ddd2b9 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:31.879505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ad00d82f-f1d2-4b1e-b8b5-3da288ddd2b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:31.938544 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00d82f-f1d2-4b1e-b8b5-3da288ddd2b9 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:31.942366 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabeef0-657c-4ef7-a9ac-b2827e37aa1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:31.942993 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabeef0-657c-4ef7-a9ac-b2827e37aa1e service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:31.943270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabeef0-657c-4ef7-a9ac-b2827e37aa1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:31.943539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabeef0-657c-4ef7-a9ac-b2827e37aa1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:32.001420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daabeef0-657c-4ef7-a9ac-b2827e37aa1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:32.002914 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daabeef0-657c-4ef7-a9ac-b2827e37aa1e service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:32.012856 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1267/2526] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:32.819004 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ad9f7-15af-4613-8a3c-48463a946562 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:32.820894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-659ad9f7-15af-4613-8a3c-48463a946562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:32.907796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ad9f7-15af-4613-8a3c-48463a946562 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:32.913481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa221fb-d77e-4e96-9376-0f5ad69af617 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:32.914100 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa221fb-d77e-4e96-9376-0f5ad69af617 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:32.914372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa221fb-d77e-4e96-9376-0f5ad69af617 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:32.914639 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa221fb-d77e-4e96-9376-0f5ad69af617 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:32.916379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-6aa221fb-d77e-4e96-9376-0f5ad69af617 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:32.929075 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa221fb-d77e-4e96-9376-0f5ad69af617 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:32.930680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa221fb-d77e-4e96-9376-0f5ad69af617 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:32.941930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1260/2527] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:32:36.345483 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabeef0-657c-4ef7-a9ac-b2827e37aa1e service cinder] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:36.347509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-daabeef0-657c-4ef7-a9ac-b2827e37aa1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:36.364536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabeef0-657c-4ef7-a9ac-b2827e37aa1e service cinder] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:36.369251 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86eb971-f0df-476c-9971-b290a68869eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:36.369860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86eb971-f0df-476c-9971-b290a68869eb service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:36.370138 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86eb971-f0df-476c-9971-b290a68869eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:36.370410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86eb971-f0df-476c-9971-b290a68869eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:36.380624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86eb971-f0df-476c-9971-b290a68869eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:36.382082 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86eb971-f0df-476c-9971-b290a68869eb service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:36.391802 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1268/2528] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:39.041609 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a311a7-0e30-409b-9dda-5054167e5026 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:39.042169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a311a7-0e30-409b-9dda-5054167e5026 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:39.042169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a311a7-0e30-409b-9dda-5054167e5026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:39.062174 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-39a311a7-0e30-409b-9dda-5054167e5026 None None] MFA Rules not processed for user `de73f338e6ef4844979ead86ef2bd16c`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:39.063424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-39a311a7-0e30-409b-9dda-5054167e5026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:39.120713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1261/2529] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:41.436276 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86eb971-f0df-476c-9971-b290a68869eb service cinder] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:41.440632 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:41.506894 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86eb971-f0df-476c-9971-b290a68869eb service cinder] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:41.511728 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99342dd6-9e26-4515-a6a7-0aa12f95ee73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:41.512469 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99342dd6-9e26-4515-a6a7-0aa12f95ee73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:41.512735 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99342dd6-9e26-4515-a6a7-0aa12f95ee73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:41.512989 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99342dd6-9e26-4515-a6a7-0aa12f95ee73 admin admin] PATH_INFO: `/v3/users/8ed6a5980077441fb60aef738726663b` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:41.514237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99342dd6-9e26-4515-a6a7-0aa12f95ee73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ed6a5980077441fb60aef738726663b)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:41.516237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99342dd6-9e26-4515-a6a7-0aa12f95ee73 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:41.575531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-99342dd6-9e26-4515-a6a7-0aa12f95ee73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ed6a5980077441fb60aef738726663b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:41.589678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-99342dd6-9e26-4515-a6a7-0aa12f95ee73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ed6a5980077441fb60aef738726663b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:41.598922 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-99342dd6-9e26-4515-a6a7-0aa12f95ee73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ed6a5980077441fb60aef738726663b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:41.600265 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-99342dd6-9e26-4515-a6a7-0aa12f95ee73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ed6a5980077441fb60aef738726663b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:41.601135 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-99342dd6-9e26-4515-a6a7-0aa12f95ee73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ed6a5980077441fb60aef738726663b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:41.622427 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1269/2530] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:41 2022] DELETE /identity/v3/users/8ed6a5980077441fb60aef738726663b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:41.627308 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364b251-5ad6-4dd0-ac01-36d1c527da5d None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:41.627790 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364b251-5ad6-4dd0-ac01-36d1c527da5d None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:41.627918 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b364b251-5ad6-4dd0-ac01-36d1c527da5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:41.648974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-b364b251-5ad6-4dd0-ac01-36d1c527da5d None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:41.650834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b364b251-5ad6-4dd0-ac01-36d1c527da5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:41.717957 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:41.736395 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1262/2531] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:41.738855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99342dd6-9e26-4515-a6a7-0aa12f95ee73 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:41.740396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-99342dd6-9e26-4515-a6a7-0aa12f95ee73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:41.805010 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99342dd6-9e26-4515-a6a7-0aa12f95ee73 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:41.808919 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ceb22-4f0b-4dbc-9b55-31e8c45a6bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:41.809664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ceb22-4f0b-4dbc-9b55-31e8c45a6bbb service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:41.809957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ceb22-4f0b-4dbc-9b55-31e8c45a6bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:41.810319 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ceb22-4f0b-4dbc-9b55-31e8c45a6bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:41.821909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ceb22-4f0b-4dbc-9b55-31e8c45a6bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:41.823335 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ceb22-4f0b-4dbc-9b55-31e8c45a6bbb service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:41.834529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1270/2532] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:32:42.426871 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364b251-5ad6-4dd0-ac01-36d1c527da5d None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:42.429080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b364b251-5ad6-4dd0-ac01-36d1c527da5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:42.441965 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b364b251-5ad6-4dd0-ac01-36d1c527da5d None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:42.447392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:42.448444 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:42.448932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:42.449427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b admin admin] PATH_INFO: `/v3/projects/f855ee64a5bb488fb18c74551fc6ca05` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:42.451276 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f855ee64a5bb488fb18c74551fc6ca05)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:42.453720 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:42.524798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f855ee64a5bb488fb18c74551fc6ca05', 'request_id': 'req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:42.532812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f855ee64a5bb488fb18c74551fc6ca05', 'request_id': 'req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:42.541056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f855ee64a5bb488fb18c74551fc6ca05', 'request_id': 'req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:42.575846 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b admin admin] The token cache is being invalidate because project f855ee64a5bb488fb18c74551fc6ca05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:42.576478 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:42.577446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:42.593635 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1263/2533] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:42 2022] DELETE /identity/v3/projects/f855ee64a5bb488fb18c74551fc6ca05 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:42.598071 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ceb22-4f0b-4dbc-9b55-31e8c45a6bbb service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:42.615903 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ceb22-4f0b-4dbc-9b55-31e8c45a6bbb service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:42.663278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:42.721682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95dd8b0-8b88-4e6d-a5b4-f21b9a5fbae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:42.721682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dd8b0-8b88-4e6d-a5b4-f21b9a5fbae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:42.721682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dd8b0-8b88-4e6d-a5b4-f21b9a5fbae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:42.721682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95dd8b0-8b88-4e6d-a5b4-f21b9a5fbae2 admin admin] PATH_INFO: `/v3/users/95be2aeac83a4e818df54809ece3099c` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:42.724017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95dd8b0-8b88-4e6d-a5b4-f21b9a5fbae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95be2aeac83a4e818df54809ece3099c)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:42.727748 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95dd8b0-8b88-4e6d-a5b4-f21b9a5fbae2 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:42.789413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-c95dd8b0-8b88-4e6d-a5b4-f21b9a5fbae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95be2aeac83a4e818df54809ece3099c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:42.803599 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-c95dd8b0-8b88-4e6d-a5b4-f21b9a5fbae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95be2aeac83a4e818df54809ece3099c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:42.811888 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-c95dd8b0-8b88-4e6d-a5b4-f21b9a5fbae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95be2aeac83a4e818df54809ece3099c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:42.813391 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-c95dd8b0-8b88-4e6d-a5b4-f21b9a5fbae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95be2aeac83a4e818df54809ece3099c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:42.814215 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-c95dd8b0-8b88-4e6d-a5b4-f21b9a5fbae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95be2aeac83a4e818df54809ece3099c'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:42.832385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1271/2534] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:42 2022] DELETE /identity/v3/users/95be2aeac83a4e818df54809ece3099c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:43.322615 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:43.326172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:43.350692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8a151-de8b-49b4-ad52-d3e32dfd103b admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:43.467062 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285c8a6-dfe3-4197-a43d-0299894d0824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:43.467857 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285c8a6-dfe3-4197-a43d-0299894d0824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:43.468515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285c8a6-dfe3-4197-a43d-0299894d0824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:43.468792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285c8a6-dfe3-4197-a43d-0299894d0824 admin admin] PATH_INFO: `/v3/projects/e6be2831107547dfb8c339a761ef6fac` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:43.470740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285c8a6-dfe3-4197-a43d-0299894d0824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6be2831107547dfb8c339a761ef6fac)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:43.473150 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285c8a6-dfe3-4197-a43d-0299894d0824 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:43.538813 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d285c8a6-dfe3-4197-a43d-0299894d0824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6be2831107547dfb8c339a761ef6fac', 'request_id': 'req-d285c8a6-dfe3-4197-a43d-0299894d0824'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:43.546364 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d285c8a6-dfe3-4197-a43d-0299894d0824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6be2831107547dfb8c339a761ef6fac', 'request_id': 'req-d285c8a6-dfe3-4197-a43d-0299894d0824'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:43.556039 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d285c8a6-dfe3-4197-a43d-0299894d0824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6be2831107547dfb8c339a761ef6fac', 'request_id': 'req-d285c8a6-dfe3-4197-a43d-0299894d0824'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:43.570529 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d285c8a6-dfe3-4197-a43d-0299894d0824 admin admin] The token cache is being invalidate because project e6be2831107547dfb8c339a761ef6fac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:43.571265 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d285c8a6-dfe3-4197-a43d-0299894d0824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:43.572085 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-d285c8a6-dfe3-4197-a43d-0299894d0824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:43.574052 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1264/2535] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:43 2022] DELETE /identity/v3/projects/e6be2831107547dfb8c339a761ef6fac => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:43.580091 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f478183-c09f-4411-ae36-356563e1a017 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:43.580329 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f478183-c09f-4411-ae36-356563e1a017 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:43.581105 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f478183-c09f-4411-ae36-356563e1a017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:43.604627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-2f478183-c09f-4411-ae36-356563e1a017 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:43.605942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-2f478183-c09f-4411-ae36-356563e1a017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:43.686089 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1272/2536] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:43.695078 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a6ba2-e88e-4094-9f3b-1e34f768484e None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:43.695392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a6ba2-e88e-4094-9f3b-1e34f768484e None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:43.695671 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a6ba2-e88e-4094-9f3b-1e34f768484e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:43.723311 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-c57a6ba2-e88e-4094-9f3b-1e34f768484e None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:43.724841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c57a6ba2-e88e-4094-9f3b-1e34f768484e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:43.736966 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1265/2537] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:43.741248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f478183-c09f-4411-ae36-356563e1a017 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:43.754388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f478183-c09f-4411-ae36-356563e1a017 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:43.767257 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298242e-b676-4e48-890c-4e3c58d8f122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:43.768097 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298242e-b676-4e48-890c-4e3c58d8f122 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:43.769915 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298242e-b676-4e48-890c-4e3c58d8f122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:43.770382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4298242e-b676-4e48-890c-4e3c58d8f122 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:43.774420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298242e-b676-4e48-890c-4e3c58d8f122 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:32:43.774809 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298242e-b676-4e48-890c-4e3c58d8f122 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:43.778668 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4298242e-b676-4e48-890c-4e3c58d8f122 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:43.801598 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1273/2538] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:32:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:32:43.802248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a6ba2-e88e-4094-9f3b-1e34f768484e None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:43.814079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a6ba2-e88e-4094-9f3b-1e34f768484e None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:43.821377 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51190b-f3a0-46da-8f3e-cbe62c3a7db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:43.822081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51190b-f3a0-46da-8f3e-cbe62c3a7db8 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:43.822567 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51190b-f3a0-46da-8f3e-cbe62c3a7db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:43.822964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51190b-f3a0-46da-8f3e-cbe62c3a7db8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:43.823965 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51190b-f3a0-46da-8f3e-cbe62c3a7db8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:43.827571 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51190b-f3a0-46da-8f3e-cbe62c3a7db8 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:43.834114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:43.885512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1266/2539] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:43 2022] POST /identity/v3/projects => generated 375 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:32:43.886676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298242e-b676-4e48-890c-4e3c58d8f122 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:43.898451 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4298242e-b676-4e48-890c-4e3c58d8f122 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:43.937022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:43.987211 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827b019-88af-4c9c-bdc1-ad3efc373340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:43.987850 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827b019-88af-4c9c-bdc1-ad3efc373340 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:43.988294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827b019-88af-4c9c-bdc1-ad3efc373340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:43.988565 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827b019-88af-4c9c-bdc1-ad3efc373340 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:43.989286 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827b019-88af-4c9c-bdc1-ad3efc373340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:43.991483 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827b019-88af-4c9c-bdc1-ad3efc373340 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:44.024070 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1274/2540] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:43 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:32:44.028032 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51190b-f3a0-46da-8f3e-cbe62c3a7db8 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:44.039848 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51190b-f3a0-46da-8f3e-cbe62c3a7db8 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:44.049665 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de5ad8-97cc-4964-a491-d2adbabf8f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:44.051070 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de5ad8-97cc-4964-a491-d2adbabf8f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:44.051353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de5ad8-97cc-4964-a491-d2adbabf8f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:44.051985 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de5ad8-97cc-4964-a491-d2adbabf8f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:44.052595 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de5ad8-97cc-4964-a491-d2adbabf8f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:44.054487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de5ad8-97cc-4964-a491-d2adbabf8f5a admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:44.067001 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1267/2541] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:44.071355 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827b019-88af-4c9c-bdc1-ad3efc373340 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:44.082779 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827b019-88af-4c9c-bdc1-ad3efc373340 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:44.089043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15da627-e770-44d6-a1d2-cae52bbc6923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:44.089689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15da627-e770-44d6-a1d2-cae52bbc6923 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:44.089964 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15da627-e770-44d6-a1d2-cae52bbc6923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:44.090221 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15da627-e770-44d6-a1d2-cae52bbc6923 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:44.090894 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15da627-e770-44d6-a1d2-cae52bbc6923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:44.092626 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15da627-e770-44d6-a1d2-cae52bbc6923 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:44.103904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1275/2542] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:44.108403 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de5ad8-97cc-4964-a491-d2adbabf8f5a admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:44.119197 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de5ad8-97cc-4964-a491-d2adbabf8f5a admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:44.126000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c231fa-787c-4fcd-9930-1b4f4ab8fb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:44.126906 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c231fa-787c-4fcd-9930-1b4f4ab8fb7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:44.127204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c231fa-787c-4fcd-9930-1b4f4ab8fb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:44.127573 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c231fa-787c-4fcd-9930-1b4f4ab8fb7b admin admin] PATH_INFO: `/v3/projects/376e152b6a5a40f7b7056ed25814d6cf/users/8e5a6b84a2cb4147bca6ccf940505a94/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:44.145967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c231fa-787c-4fcd-9930-1b4f4ab8fb7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=376e152b6a5a40f7b7056ed25814d6cf, user_id=8e5a6b84a2cb4147bca6ccf940505a94, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:44.148810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c231fa-787c-4fcd-9930-1b4f4ab8fb7b admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:44.165584 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1268/2543] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:44 2022] PUT /identity/v3/projects/376e152b6a5a40f7b7056ed25814d6cf/users/8e5a6b84a2cb4147bca6ccf940505a94/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:44.179567 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411498e-67cc-4569-b768-64dd6cfa9e7d None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:44.179894 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411498e-67cc-4569-b768-64dd6cfa9e7d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:44.180234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411498e-67cc-4569-b768-64dd6cfa9e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:44.212456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-f411498e-67cc-4569-b768-64dd6cfa9e7d None None] MFA Rules not processed for user `8e5a6b84a2cb4147bca6ccf940505a94`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:44.213781 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f411498e-67cc-4569-b768-64dd6cfa9e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:44.287880 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1276/2544] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:44.300264 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c231fa-787c-4fcd-9930-1b4f4ab8fb7b admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:44.302207 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-d6c231fa-787c-4fcd-9930-1b4f4ab8fb7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:44.370129 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c231fa-787c-4fcd-9930-1b4f4ab8fb7b admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:44.374468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8bf45b-d73a-4519-8907-90b3dba7449f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9daa1a3c7654dcfa5ccb03f467ce7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:44.375115 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bf45b-d73a-4519-8907-90b3dba7449f service glance] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:44.375506 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bf45b-d73a-4519-8907-90b3dba7449f service glance] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:44.375654 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bf45b-d73a-4519-8907-90b3dba7449f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:44.385668 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8bf45b-d73a-4519-8907-90b3dba7449f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:44.387107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8bf45b-d73a-4519-8907-90b3dba7449f service glance] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:44.398324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1269/2545] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:48.610973 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b438b4-8c2e-4f46-8095-1e1d7a617d5c None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:48.611576 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b438b4-8c2e-4f46-8095-1e1d7a617d5c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:48.611576 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b438b4-8c2e-4f46-8095-1e1d7a617d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:48.631168 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-26b438b4-8c2e-4f46-8095-1e1d7a617d5c None None] MFA Rules not processed for user `61ff29258a214817b8fc15b51fe8b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:48.632453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-26b438b4-8c2e-4f46-8095-1e1d7a617d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:48.697186 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1277/2546] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Aug 7 19:32:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:48.750160 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8bf45b-d73a-4519-8907-90b3dba7449f service glance] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:48.751647 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-2f8bf45b-d73a-4519-8907-90b3dba7449f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:48.847329 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8bf45b-d73a-4519-8907-90b3dba7449f service glance] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:48.854344 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87833bb3-16ea-4304-b1e1-afc89d09583a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db51971158a946c98a8ed716ecc5d72b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:48.855230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87833bb3-16ea-4304-b1e1-afc89d09583a service swift] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:48.855538 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87833bb3-16ea-4304-b1e1-afc89d09583a service swift] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:48.855890 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87833bb3-16ea-4304-b1e1-afc89d09583a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:48.866159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87833bb3-16ea-4304-b1e1-afc89d09583a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:48.867801 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87833bb3-16ea-4304-b1e1-afc89d09583a service swift] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:48.903242 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1270/2547] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Aug 7 19:32:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 19:32:49.484808 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b438b4-8c2e-4f46-8095-1e1d7a617d5c None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:49.486332 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-26b438b4-8c2e-4f46-8095-1e1d7a617d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:49.607622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:49.630446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b438b4-8c2e-4f46-8095-1e1d7a617d5c None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:49.639742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fd639-9fdc-4559-8250-2d76be76586e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:49.640407 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fd639-9fdc-4559-8250-2d76be76586e service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:49.640690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fd639-9fdc-4559-8250-2d76be76586e service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:49.640965 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fd639-9fdc-4559-8250-2d76be76586e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:49.648005 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d56fd639-9fdc-4559-8250-2d76be76586e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:49.759651 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fd639-9fdc-4559-8250-2d76be76586e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:49.761502 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fd639-9fdc-4559-8250-2d76be76586e service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:49.780478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1278/2548] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:32:55.014195 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87833bb3-16ea-4304-b1e1-afc89d09583a service swift] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:55.016711 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-87833bb3-16ea-4304-b1e1-afc89d09583a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:55.082560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:55.143764 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87833bb3-16ea-4304-b1e1-afc89d09583a service swift] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:55.153583 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7147c-f96c-4918-adef-4e2b2311199a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '143c30b4201645279564e72ab3288adc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:55.154358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7147c-f96c-4918-adef-4e2b2311199a service placement] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:55.154718 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7147c-f96c-4918-adef-4e2b2311199a service placement] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:55.154907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7147c-f96c-4918-adef-4e2b2311199a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:55.156830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-14e7147c-f96c-4918-adef-4e2b2311199a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:55.162774 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fd639-9fdc-4559-8250-2d76be76586e service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:55.164424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d56fd639-9fdc-4559-8250-2d76be76586e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:55.182038 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fd639-9fdc-4559-8250-2d76be76586e service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:55.190550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411588e-2c42-48cc-b9a0-84912265fb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:55.191981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411588e-2c42-48cc-b9a0-84912265fb02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:55.192517 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411588e-2c42-48cc-b9a0-84912265fb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:55.192993 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411588e-2c42-48cc-b9a0-84912265fb02 admin admin] PATH_INFO: `/v3/users/be3e2afc428a45e2b993c39e3d53cd8b` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:55.193236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7147c-f96c-4918-adef-4e2b2311199a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:55.194666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411588e-2c42-48cc-b9a0-84912265fb02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be3e2afc428a45e2b993c39e3d53cd8b)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:55.195586 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7147c-f96c-4918-adef-4e2b2311199a service placement] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:55.197375 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411588e-2c42-48cc-b9a0-84912265fb02 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:55.218436 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1271/2549] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:55.281448 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-2411588e-2c42-48cc-b9a0-84912265fb02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be3e2afc428a45e2b993c39e3d53cd8b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:55.309781 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-2411588e-2c42-48cc-b9a0-84912265fb02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be3e2afc428a45e2b993c39e3d53cd8b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:55.329172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-2411588e-2c42-48cc-b9a0-84912265fb02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be3e2afc428a45e2b993c39e3d53cd8b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:55.329172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-2411588e-2c42-48cc-b9a0-84912265fb02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be3e2afc428a45e2b993c39e3d53cd8b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:55.329587 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-2411588e-2c42-48cc-b9a0-84912265fb02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be3e2afc428a45e2b993c39e3d53cd8b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:55.372988 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1279/2550] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:32:55 2022] DELETE /identity/v3/users/be3e2afc428a45e2b993c39e3d53cd8b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:32:55.762629 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7147c-f96c-4918-adef-4e2b2311199a service placement] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:55.784074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7147c-f96c-4918-adef-4e2b2311199a service placement] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:55.917182 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf325fa0-e466-42f8-9fa2-383dfaea49a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:55.917182 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf325fa0-e466-42f8-9fa2-383dfaea49a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:55.917708 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf325fa0-e466-42f8-9fa2-383dfaea49a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:55.918286 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf325fa0-e466-42f8-9fa2-383dfaea49a1 admin admin] PATH_INFO: `/v3/projects/b10e8224d7aa4b019ccad26aa3398933` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:55.919563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf325fa0-e466-42f8-9fa2-383dfaea49a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b10e8224d7aa4b019ccad26aa3398933)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:55.922063 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf325fa0-e466-42f8-9fa2-383dfaea49a1 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:56.034748 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cf325fa0-e466-42f8-9fa2-383dfaea49a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b10e8224d7aa4b019ccad26aa3398933', 'request_id': 'req-cf325fa0-e466-42f8-9fa2-383dfaea49a1'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:56.062071 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cf325fa0-e466-42f8-9fa2-383dfaea49a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b10e8224d7aa4b019ccad26aa3398933', 'request_id': 'req-cf325fa0-e466-42f8-9fa2-383dfaea49a1'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:56.071382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cf325fa0-e466-42f8-9fa2-383dfaea49a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b10e8224d7aa4b019ccad26aa3398933', 'request_id': 'req-cf325fa0-e466-42f8-9fa2-383dfaea49a1'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:56.101914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cf325fa0-e466-42f8-9fa2-383dfaea49a1 admin admin] The token cache is being invalidate because project b10e8224d7aa4b019ccad26aa3398933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:32:56.102553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cf325fa0-e466-42f8-9fa2-383dfaea49a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:56.103481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cf325fa0-e466-42f8-9fa2-383dfaea49a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:32:56.119498 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1272/2551] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:32:55 2022] DELETE /identity/v3/projects/b10e8224d7aa4b019ccad26aa3398933 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:56.125327 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982a480-2a8e-4898-9559-2782e130e409 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:56.125908 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982a480-2a8e-4898-9559-2782e130e409 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:56.126475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982a480-2a8e-4898-9559-2782e130e409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:56.171226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-6982a480-2a8e-4898-9559-2782e130e409 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:56.173849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6982a480-2a8e-4898-9559-2782e130e409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:56.276311 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1280/2552] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:56.279571 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9bcac-4d4a-4757-a80b-ec285788969e None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:56.279891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9bcac-4d4a-4757-a80b-ec285788969e None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:56.280185 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9bcac-4d4a-4757-a80b-ec285788969e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:56.311335 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-6ed9bcac-4d4a-4757-a80b-ec285788969e None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:56.312603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-6ed9bcac-4d4a-4757-a80b-ec285788969e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:56.324441 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1273/2553] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:56.326519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982a480-2a8e-4898-9559-2782e130e409 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:56.337033 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982a480-2a8e-4898-9559-2782e130e409 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:56.342313 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac3aab-45c1-48be-974a-bb8a2ba45ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:56.342930 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac3aab-45c1-48be-974a-bb8a2ba45ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:56.343443 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac3aab-45c1-48be-974a-bb8a2ba45ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:56.343578 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac3aab-45c1-48be-974a-bb8a2ba45ec5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:56.344772 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac3aab-45c1-48be-974a-bb8a2ba45ec5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:32:56.344895 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac3aab-45c1-48be-974a-bb8a2ba45ec5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:56.346611 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac3aab-45c1-48be-974a-bb8a2ba45ec5 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:56.349421 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:56.368979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9bcac-4d4a-4757-a80b-ec285788969e None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:56.376281 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1281/2554] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:32:56.379527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9bcac-4d4a-4757-a80b-ec285788969e None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:56.384363 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71d0a4-9544-494b-bcfb-9a0f3649bfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:56.385008 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71d0a4-9544-494b-bcfb-9a0f3649bfe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:56.385273 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71d0a4-9544-494b-bcfb-9a0f3649bfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:56.385558 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71d0a4-9544-494b-bcfb-9a0f3649bfe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:56.386206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71d0a4-9544-494b-bcfb-9a0f3649bfe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:56.388280 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71d0a4-9544-494b-bcfb-9a0f3649bfe8 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:56.427034 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1274/2555] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:56 2022] POST /identity/v3/projects => generated 357 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:56.431862 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac3aab-45c1-48be-974a-bb8a2ba45ec5 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:56.442988 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac3aab-45c1-48be-974a-bb8a2ba45ec5 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:56.533062 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab869c2-2195-4d58-a93c-76357e55082f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:56.534133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab869c2-2195-4d58-a93c-76357e55082f admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:56.534509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab869c2-2195-4d58-a93c-76357e55082f admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:56.534890 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab869c2-2195-4d58-a93c-76357e55082f admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:56.535616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab869c2-2195-4d58-a93c-76357e55082f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:56.538639 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab869c2-2195-4d58-a93c-76357e55082f admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:56.575008 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1282/2556] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:56 2022] POST /identity/v3/users => generated 279 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:56.577428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71d0a4-9544-494b-bcfb-9a0f3649bfe8 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:56.592757 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71d0a4-9544-494b-bcfb-9a0f3649bfe8 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:56.604816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273408ad-7878-4a29-8cc0-96f292c1b6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:56.605479 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273408ad-7878-4a29-8cc0-96f292c1b6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:56.605734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273408ad-7878-4a29-8cc0-96f292c1b6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:56.606006 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273408ad-7878-4a29-8cc0-96f292c1b6d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:56.606832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273408ad-7878-4a29-8cc0-96f292c1b6d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:56.608569 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273408ad-7878-4a29-8cc0-96f292c1b6d8 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:56.623575 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1275/2557] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:56.624395 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab869c2-2195-4d58-a93c-76357e55082f admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:56.650330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab869c2-2195-4d58-a93c-76357e55082f admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:56.662150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f245d-c2df-41fb-8649-0171cd85dc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:56.663274 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f245d-c2df-41fb-8649-0171cd85dc91 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:56.663888 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f245d-c2df-41fb-8649-0171cd85dc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:56.664353 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f245d-c2df-41fb-8649-0171cd85dc91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:56.665202 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f245d-c2df-41fb-8649-0171cd85dc91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:56.667022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f245d-c2df-41fb-8649-0171cd85dc91 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:56.688462 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1283/2558] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:56.693816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273408ad-7878-4a29-8cc0-96f292c1b6d8 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:56.710737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273408ad-7878-4a29-8cc0-96f292c1b6d8 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:56.716496 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3593efea-8a22-4d1f-8efa-7faf99fe3ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:56.717404 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593efea-8a22-4d1f-8efa-7faf99fe3ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:56.717783 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593efea-8a22-4d1f-8efa-7faf99fe3ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:56.718159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593efea-8a22-4d1f-8efa-7faf99fe3ba8 admin admin] PATH_INFO: `/v3/projects/1f61c12cffca44f1a1cbef811b38b23b/users/1ea903d5ae1f4ad88f1fa12c29642370/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:56.735894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3593efea-8a22-4d1f-8efa-7faf99fe3ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f61c12cffca44f1a1cbef811b38b23b, user_id=1ea903d5ae1f4ad88f1fa12c29642370, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:56.739211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3593efea-8a22-4d1f-8efa-7faf99fe3ba8 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:56.750428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:56.757950 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1276/2559] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:56 2022] PUT /identity/v3/projects/1f61c12cffca44f1a1cbef811b38b23b/users/1ea903d5ae1f4ad88f1fa12c29642370/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:32:56.768730 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7f40c-cbf8-45fb-b44f-6eecd52dce4d None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:56.769307 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7f40c-cbf8-45fb-b44f-6eecd52dce4d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:56.769836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e7f40c-cbf8-45fb-b44f-6eecd52dce4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:56.812416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-a6e7f40c-cbf8-45fb-b44f-6eecd52dce4d None None] MFA Rules not processed for user `1ea903d5ae1f4ad88f1fa12c29642370`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:56.813644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-a6e7f40c-cbf8-45fb-b44f-6eecd52dce4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:56.907536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1284/2560] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:56.912624 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3593efea-8a22-4d1f-8efa-7faf99fe3ba8 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:57.003424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3593efea-8a22-4d1f-8efa-7faf99fe3ba8 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:57.011376 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d1ae2-ca0c-4892-9473-814047ca38fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:57.012132 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d1ae2-ca0c-4892-9473-814047ca38fc admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:57.012732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d1ae2-ca0c-4892-9473-814047ca38fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:57.012910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d1ae2-ca0c-4892-9473-814047ca38fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:57.013706 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d1ae2-ca0c-4892-9473-814047ca38fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:57.017443 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d1ae2-ca0c-4892-9473-814047ca38fc admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:57.071121 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1277/2561] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:32:56 2022] POST /identity/v3/projects => generated 355 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:32:57.075665 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7f40c-cbf8-45fb-b44f-6eecd52dce4d None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:57.087763 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e7f40c-cbf8-45fb-b44f-6eecd52dce4d None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:57.160095 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f25cc8-a6c6-40c4-937e-7c954c9987a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:57.160900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f25cc8-a6c6-40c4-937e-7c954c9987a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:57.161186 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f25cc8-a6c6-40c4-937e-7c954c9987a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:57.161507 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f25cc8-a6c6-40c4-937e-7c954c9987a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:57.162243 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f25cc8-a6c6-40c4-937e-7c954c9987a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:57.164455 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f25cc8-a6c6-40c4-937e-7c954c9987a2 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:57.176536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:57.198487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d1ae2-ca0c-4892-9473-814047ca38fc admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:57.204724 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1285/2562] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:32:57 2022] POST /identity/v3/users => generated 284 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:32:57.212404 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d1ae2-ca0c-4892-9473-814047ca38fc admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:57.222509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6d330-9f6f-437b-9a1e-1256d8e47e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:57.223269 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6d330-9f6f-437b-9a1e-1256d8e47e23 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:57.223553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6d330-9f6f-437b-9a1e-1256d8e47e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:57.224057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6d330-9f6f-437b-9a1e-1256d8e47e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:57.224730 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6d330-9f6f-437b-9a1e-1256d8e47e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:57.226509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6d330-9f6f-437b-9a1e-1256d8e47e23 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:57.241787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1278/2563] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:32:57.246540 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f25cc8-a6c6-40c4-937e-7c954c9987a2 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:57.261915 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f25cc8-a6c6-40c4-937e-7c954c9987a2 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:57.267115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac0774-a387-480b-9ed4-bc3bd7a4eb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:57.267942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac0774-a387-480b-9ed4-bc3bd7a4eb63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:57.268263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac0774-a387-480b-9ed4-bc3bd7a4eb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:57.268674 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac0774-a387-480b-9ed4-bc3bd7a4eb63 admin admin] PATH_INFO: `/v3/projects/27cf1027dd8b4747b707ba9269d29a47/users/0f1faf903753454fb1b8015071ec7f2d/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:57.288691 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac0774-a387-480b-9ed4-bc3bd7a4eb63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27cf1027dd8b4747b707ba9269d29a47, user_id=0f1faf903753454fb1b8015071ec7f2d, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:57.291590 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac0774-a387-480b-9ed4-bc3bd7a4eb63 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:57.312153 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1286/2564] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:57 2022] PUT /identity/v3/projects/27cf1027dd8b4747b707ba9269d29a47/users/0f1faf903753454fb1b8015071ec7f2d/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:32:57.317770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6d330-9f6f-437b-9a1e-1256d8e47e23 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:57.329244 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6d330-9f6f-437b-9a1e-1256d8e47e23 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:57.481776 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8646127-00e5-4d37-a8b7-20f8cfd3757e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:57.483081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8646127-00e5-4d37-a8b7-20f8cfd3757e admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:57.483610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8646127-00e5-4d37-a8b7-20f8cfd3757e admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:57.484172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8646127-00e5-4d37-a8b7-20f8cfd3757e admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:57.485376 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8646127-00e5-4d37-a8b7-20f8cfd3757e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:57.488368 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8646127-00e5-4d37-a8b7-20f8cfd3757e admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:57.517413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac0774-a387-480b-9ed4-bc3bd7a4eb63 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:57.518163 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1279/2565] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:32:57.540629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac0774-a387-480b-9ed4-bc3bd7a4eb63 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:57.543872 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83909df9-27da-4b80-abac-8429098dd904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:57.544544 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83909df9-27da-4b80-abac-8429098dd904 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:57.545346 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83909df9-27da-4b80-abac-8429098dd904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:57.545694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83909df9-27da-4b80-abac-8429098dd904 admin admin] PATH_INFO: `/v3/projects/27cf1027dd8b4747b707ba9269d29a47/users/0f1faf903753454fb1b8015071ec7f2d/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:57.548105 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83909df9-27da-4b80-abac-8429098dd904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27cf1027dd8b4747b707ba9269d29a47, user_id=0f1faf903753454fb1b8015071ec7f2d, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:57.551615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83909df9-27da-4b80-abac-8429098dd904 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:57.572717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1287/2566] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:32:57 2022] PUT /identity/v3/projects/27cf1027dd8b4747b707ba9269d29a47/users/0f1faf903753454fb1b8015071ec7f2d/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:32:57.586104 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae5610-ac67-444c-9222-57906081018f None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:57.586420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae5610-ac67-444c-9222-57906081018f None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:57.586671 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ae5610-ac67-444c-9222-57906081018f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:57.623745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-43ae5610-ac67-444c-9222-57906081018f None None] MFA Rules not processed for user `0f1faf903753454fb1b8015071ec7f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:32:57.627422 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-43ae5610-ac67-444c-9222-57906081018f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:57.699084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:32:57.716053 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1280/2567] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:32:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:32:57.726535 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83909df9-27da-4b80-abac-8429098dd904 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:57.728330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-83909df9-27da-4b80-abac-8429098dd904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:57.822769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83909df9-27da-4b80-abac-8429098dd904 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:57.830992 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb5fbc-1a03-47f0-ac1b-3f88ba81d196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:57.831639 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb5fbc-1a03-47f0-ac1b-3f88ba81d196 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:57.831917 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb5fbc-1a03-47f0-ac1b-3f88ba81d196 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:57.832562 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb5fbc-1a03-47f0-ac1b-3f88ba81d196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:57.841131 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae5610-ac67-444c-9222-57906081018f None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:57.842372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-43ae5610-ac67-444c-9222-57906081018f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:57.844440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb5fbc-1a03-47f0-ac1b-3f88ba81d196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:57.848102 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb5fbc-1a03-47f0-ac1b-3f88ba81d196 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:57.861168 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1288/2568] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:32:57.977234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb5fbc-1a03-47f0-ac1b-3f88ba81d196 service cinder] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:32:57.979468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8acb5fbc-1a03-47f0-ac1b-3f88ba81d196 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:57.996574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb5fbc-1a03-47f0-ac1b-3f88ba81d196 service cinder] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:58.003890 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ae5610-ac67-444c-9222-57906081018f None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:32:58.022199 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0b23e-ec58-45cd-87ff-b16a0de2d7e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:58.022592 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63feb6e7-2c9b-4f34-b632-0703c30943c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:32:58.022741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0b23e-ec58-45cd-87ff-b16a0de2d7e7 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:58.023323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63feb6e7-2c9b-4f34-b632-0703c30943c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:32:58.023570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0b23e-ec58-45cd-87ff-b16a0de2d7e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:58.023684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63feb6e7-2c9b-4f34-b632-0703c30943c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:32:58.023858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63feb6e7-2c9b-4f34-b632-0703c30943c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:58.024595 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0b23e-ec58-45cd-87ff-b16a0de2d7e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:32:58.028840 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-32a0b23e-ec58-45cd-87ff-b16a0de2d7e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:32:58.123956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0b23e-ec58-45cd-87ff-b16a0de2d7e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:58.125866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0b23e-ec58-45cd-87ff-b16a0de2d7e7 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:58.138129 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1281/2569] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:32:58.185609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63feb6e7-2c9b-4f34-b632-0703c30943c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:32:58.189937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63feb6e7-2c9b-4f34-b632-0703c30943c3 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:32:58.212720 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1289/2570] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:32:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:33:02.888478 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e747b1-2e8b-4b0f-90d3-a91b90cfc9fc None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:02.889017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e747b1-2e8b-4b0f-90d3-a91b90cfc9fc None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:02.889017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e747b1-2e8b-4b0f-90d3-a91b90cfc9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:02.909594 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-a6e747b1-2e8b-4b0f-90d3-a91b90cfc9fc None None] MFA Rules not processed for user `61ff29258a214817b8fc15b51fe8b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:02.910872 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a6e747b1-2e8b-4b0f-90d3-a91b90cfc9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:02.973826 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1282/2571] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Aug 7 19:33:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:02.986882 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63feb6e7-2c9b-4f34-b632-0703c30943c3 service cinder] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:02.988627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-63feb6e7-2c9b-4f34-b632-0703c30943c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:03.053272 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:03.070920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63feb6e7-2c9b-4f34-b632-0703c30943c3 service cinder] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:03.077221 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880187f2-df06-451a-8507-9f663208bf50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db51971158a946c98a8ed716ecc5d72b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:03.077849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880187f2-df06-451a-8507-9f663208bf50 service swift] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:03.078129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880187f2-df06-451a-8507-9f663208bf50 service swift] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:03.078403 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880187f2-df06-451a-8507-9f663208bf50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:03.089479 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880187f2-df06-451a-8507-9f663208bf50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:03.091466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880187f2-df06-451a-8507-9f663208bf50 service swift] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:03.101676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1290/2572] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Aug 7 19:33:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 19:33:05.008433 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e747b1-2e8b-4b0f-90d3-a91b90cfc9fc None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:05.010314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a6e747b1-2e8b-4b0f-90d3-a91b90cfc9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:05.099512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:05.143343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e747b1-2e8b-4b0f-90d3-a91b90cfc9fc None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:05.156598 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04fc501-5de6-4801-af4c-324c6b5d4c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:05.157541 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fc501-5de6-4801-af4c-324c6b5d4c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:05.157828 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fc501-5de6-4801-af4c-324c6b5d4c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:05.158215 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04fc501-5de6-4801-af4c-324c6b5d4c94 admin admin] PATH_INFO: `/v3/users/8e5a6b84a2cb4147bca6ccf940505a94` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:05.160068 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04fc501-5de6-4801-af4c-324c6b5d4c94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e5a6b84a2cb4147bca6ccf940505a94)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:05.162757 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04fc501-5de6-4801-af4c-324c6b5d4c94 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:05.261837 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-a04fc501-5de6-4801-af4c-324c6b5d4c94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e5a6b84a2cb4147bca6ccf940505a94'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:05.281610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-a04fc501-5de6-4801-af4c-324c6b5d4c94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e5a6b84a2cb4147bca6ccf940505a94'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:05.308876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-a04fc501-5de6-4801-af4c-324c6b5d4c94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e5a6b84a2cb4147bca6ccf940505a94'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:05.316664 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-a04fc501-5de6-4801-af4c-324c6b5d4c94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e5a6b84a2cb4147bca6ccf940505a94'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:05.331651 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-a04fc501-5de6-4801-af4c-324c6b5d4c94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e5a6b84a2cb4147bca6ccf940505a94'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:05.346692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1283/2573] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:33:05 2022] DELETE /identity/v3/users/8e5a6b84a2cb4147bca6ccf940505a94 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:05.348633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c829f-3887-4fca-8362-00858af9fbf6 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:05.349102 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c829f-3887-4fca-8362-00858af9fbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:05.349227 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c829f-3887-4fca-8362-00858af9fbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:05.388448 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-5e9c829f-3887-4fca-8362-00858af9fbf6 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:05.390755 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5e9c829f-3887-4fca-8362-00858af9fbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:05.501612 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1291/2574] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:05.515886 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04fc501-5de6-4801-af4c-324c6b5d4c94 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:05.523854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a04fc501-5de6-4801-af4c-324c6b5d4c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:05.662096 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04fc501-5de6-4801-af4c-324c6b5d4c94 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:05.667030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761ff1a-6698-42c0-97f0-bc3c4f5ae08f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:05.667773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761ff1a-6698-42c0-97f0-bc3c4f5ae08f service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:05.668187 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761ff1a-6698-42c0-97f0-bc3c4f5ae08f service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:05.668552 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761ff1a-6698-42c0-97f0-bc3c4f5ae08f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:05.686912 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761ff1a-6698-42c0-97f0-bc3c4f5ae08f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:05.689578 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761ff1a-6698-42c0-97f0-bc3c4f5ae08f service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:05.703836 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1284/2575] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:33:06.088119 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c829f-3887-4fca-8362-00858af9fbf6 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:06.089608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5e9c829f-3887-4fca-8362-00858af9fbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:06.126212 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c829f-3887-4fca-8362-00858af9fbf6 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:06.145212 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee5705-c83b-43f6-aff8-edbc9bf3617d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:06.145892 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee5705-c83b-43f6-aff8-edbc9bf3617d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:06.146168 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee5705-c83b-43f6-aff8-edbc9bf3617d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:06.146440 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee5705-c83b-43f6-aff8-edbc9bf3617d admin admin] PATH_INFO: `/v3/projects/376e152b6a5a40f7b7056ed25814d6cf` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:06.147790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee5705-c83b-43f6-aff8-edbc9bf3617d admin admin] RBAC: Authorizing `identity:delete_project(project_id=376e152b6a5a40f7b7056ed25814d6cf)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:06.150185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee5705-c83b-43f6-aff8-edbc9bf3617d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:06.219357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-72ee5705-c83b-43f6-aff8-edbc9bf3617d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '376e152b6a5a40f7b7056ed25814d6cf', 'request_id': 'req-72ee5705-c83b-43f6-aff8-edbc9bf3617d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:06.227297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-72ee5705-c83b-43f6-aff8-edbc9bf3617d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '376e152b6a5a40f7b7056ed25814d6cf', 'request_id': 'req-72ee5705-c83b-43f6-aff8-edbc9bf3617d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:06.245362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-72ee5705-c83b-43f6-aff8-edbc9bf3617d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '376e152b6a5a40f7b7056ed25814d6cf', 'request_id': 'req-72ee5705-c83b-43f6-aff8-edbc9bf3617d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:06.266759 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-72ee5705-c83b-43f6-aff8-edbc9bf3617d admin admin] The token cache is being invalidate because project 376e152b6a5a40f7b7056ed25814d6cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:33:06.266986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-72ee5705-c83b-43f6-aff8-edbc9bf3617d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:06.268093 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-72ee5705-c83b-43f6-aff8-edbc9bf3617d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:06.269774 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1292/2576] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:33:06 2022] DELETE /identity/v3/projects/376e152b6a5a40f7b7056ed25814d6cf => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:06.278782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb02958-9b45-40a2-9869-45bc3d28f358 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:06.279337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb02958-9b45-40a2-9869-45bc3d28f358 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:06.279680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb02958-9b45-40a2-9869-45bc3d28f358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:06.309088 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-6fb02958-9b45-40a2-9869-45bc3d28f358 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:06.310626 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-6fb02958-9b45-40a2-9869-45bc3d28f358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:06.380928 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:06.398262 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1285/2577] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:06.410732 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e41cf-3247-4206-8e5e-448a0c67571b None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:06.411657 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e41cf-3247-4206-8e5e-448a0c67571b None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:06.411657 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e41cf-3247-4206-8e5e-448a0c67571b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:06.437456 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-bd6e41cf-3247-4206-8e5e-448a0c67571b None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:06.438891 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-bd6e41cf-3247-4206-8e5e-448a0c67571b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:06.454688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1293/2578] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:06.455130 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb02958-9b45-40a2-9869-45bc3d28f358 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:06.486906 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb02958-9b45-40a2-9869-45bc3d28f358 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:06.495515 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e47bb0-bff9-4d34-b029-681a0d68f5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:06.496236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e47bb0-bff9-4d34-b029-681a0d68f5a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:06.496492 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e47bb0-bff9-4d34-b029-681a0d68f5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:06.496747 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e47bb0-bff9-4d34-b029-681a0d68f5a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:06.497428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e47bb0-bff9-4d34-b029-681a0d68f5a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:33:06.497732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e47bb0-bff9-4d34-b029-681a0d68f5a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:06.499422 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e47bb0-bff9-4d34-b029-681a0d68f5a6 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:06.519462 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1286/2579] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:33:06.523999 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e41cf-3247-4206-8e5e-448a0c67571b None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:06.533756 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e41cf-3247-4206-8e5e-448a0c67571b None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:06.539580 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94b939-f978-45f6-96a4-b962d06dcd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:06.540329 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94b939-f978-45f6-96a4-b962d06dcd67 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:06.541072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94b939-f978-45f6-96a4-b962d06dcd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:06.541072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb94b939-f978-45f6-96a4-b962d06dcd67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:06.541609 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb94b939-f978-45f6-96a4-b962d06dcd67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:06.543629 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb94b939-f978-45f6-96a4-b962d06dcd67 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:06.575907 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1294/2580] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:33:06 2022] POST /identity/v3/projects => generated 397 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:06.580436 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e47bb0-bff9-4d34-b029-681a0d68f5a6 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:06.592205 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e47bb0-bff9-4d34-b029-681a0d68f5a6 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:06.684605 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb7fe0-e6a6-472a-93fc-4ee18cb28430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:06.685239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb7fe0-e6a6-472a-93fc-4ee18cb28430 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:06.685519 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb7fe0-e6a6-472a-93fc-4ee18cb28430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:06.685830 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb7fe0-e6a6-472a-93fc-4ee18cb28430 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:06.686677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb7fe0-e6a6-472a-93fc-4ee18cb28430 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:06.688474 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb7fe0-e6a6-472a-93fc-4ee18cb28430 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:06.721447 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1287/2581] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:33:06 2022] POST /identity/v3/users => generated 299 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:06.725547 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94b939-f978-45f6-96a4-b962d06dcd67 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:06.768121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb94b939-f978-45f6-96a4-b962d06dcd67 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:06.796904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e8dd7-638a-4d4f-bbed-f31cc93483f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:06.800024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e8dd7-638a-4d4f-bbed-f31cc93483f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:06.800594 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e8dd7-638a-4d4f-bbed-f31cc93483f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:06.801029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e8dd7-638a-4d4f-bbed-f31cc93483f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:06.805834 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e8dd7-638a-4d4f-bbed-f31cc93483f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:06.809921 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e8dd7-638a-4d4f-bbed-f31cc93483f2 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:06.829910 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1295/2582] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:06.841838 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb7fe0-e6a6-472a-93fc-4ee18cb28430 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:06.849910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb7fe0-e6a6-472a-93fc-4ee18cb28430 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:06.864725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653aa095-02cc-4f33-b064-f5733d993066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:06.866942 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653aa095-02cc-4f33-b064-f5733d993066 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:06.867154 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653aa095-02cc-4f33-b064-f5733d993066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:06.868050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653aa095-02cc-4f33-b064-f5733d993066 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:06.870306 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653aa095-02cc-4f33-b064-f5733d993066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:06.874836 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653aa095-02cc-4f33-b064-f5733d993066 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:06.917012 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1288/2583] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:06.922206 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e8dd7-638a-4d4f-bbed-f31cc93483f2 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:06.935350 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e8dd7-638a-4d4f-bbed-f31cc93483f2 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:06.941294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3b39e-1e58-4e83-a982-6050ae808993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:06.942091 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3b39e-1e58-4e83-a982-6050ae808993 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:06.942386 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3b39e-1e58-4e83-a982-6050ae808993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:06.942709 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3b39e-1e58-4e83-a982-6050ae808993 admin admin] PATH_INFO: `/v3/projects/12821f57ae714187b4b7c3955468fdb5/users/3412470ced8140fe982bb964be881b58/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:06.968577 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3b39e-1e58-4e83-a982-6050ae808993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12821f57ae714187b4b7c3955468fdb5, user_id=3412470ced8140fe982bb964be881b58, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:06.971378 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c3b39e-1e58-4e83-a982-6050ae808993 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:06.989453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1296/2584] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:06 2022] PUT /identity/v3/projects/12821f57ae714187b4b7c3955468fdb5/users/3412470ced8140fe982bb964be881b58/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:07.000193 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17faa633-ac5d-4cbd-9778-1016a2e08972 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:07.000600 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17faa633-ac5d-4cbd-9778-1016a2e08972 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:07.001072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17faa633-ac5d-4cbd-9778-1016a2e08972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:07.038497 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-17faa633-ac5d-4cbd-9778-1016a2e08972 None None] MFA Rules not processed for user `3412470ced8140fe982bb964be881b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:07.040129 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-17faa633-ac5d-4cbd-9778-1016a2e08972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:07.131552 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1289/2585] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:06 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:07.152072 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3b39e-1e58-4e83-a982-6050ae808993 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:07.155020 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-30c3b39e-1e58-4e83-a982-6050ae808993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:07.167912 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:07.239342 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3b39e-1e58-4e83-a982-6050ae808993 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:07.243807 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125548a-ac64-42c2-84ea-2e0cc76869ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:07.244531 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125548a-ac64-42c2-84ea-2e0cc76869ee service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:07.244937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125548a-ac64-42c2-84ea-2e0cc76869ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:07.245267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125548a-ac64-42c2-84ea-2e0cc76869ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:07.261627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125548a-ac64-42c2-84ea-2e0cc76869ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:07.263635 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125548a-ac64-42c2-84ea-2e0cc76869ee service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:07.293079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1297/2586] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:07 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:33:07.590024 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d835ec-4763-464d-85fd-aa2d8af097f6 None None] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:07.590603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d835ec-4763-464d-85fd-aa2d8af097f6 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:07.590690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d835ec-4763-464d-85fd-aa2d8af097f6 None None] PATH_INFO: `/` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:07.592385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1290/2587] 10.222.0.17 () {54 vars in 921 bytes} [Sun Aug 7 19:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 19:33:07.599234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb905b6-2871-4deb-a3c2-284ab245162f None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:07.599772 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb905b6-2871-4deb-a3c2-284ab245162f None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:07.599772 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb905b6-2871-4deb-a3c2-284ab245162f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:07.649969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-dbb905b6-2871-4deb-a3c2-284ab245162f None None] MFA Rules not processed for user `a4946aec299f45bd8402cff3159d04d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:07.651245 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-dbb905b6-2871-4deb-a3c2-284ab245162f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:07.793625 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1298/2588] 10.222.0.17 () {58 vars in 1018 bytes} [Sun Aug 7 19:33:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:07.834343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d835ec-4763-464d-85fd-aa2d8af097f6 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:07.836351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b8d835ec-4763-464d-85fd-aa2d8af097f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:07.943026 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d835ec-4763-464d-85fd-aa2d8af097f6 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:07.947567 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17b24b-a807-4240-893f-0cf2dda936cd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9daa1a3c7654dcfa5ccb03f467ce7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:07.948517 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17b24b-a807-4240-893f-0cf2dda936cd service glance] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:07.948906 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17b24b-a807-4240-893f-0cf2dda936cd service glance] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:07.949239 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17b24b-a807-4240-893f-0cf2dda936cd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:07.958219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:07.961025 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17b24b-a807-4240-893f-0cf2dda936cd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:07.962797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17b24b-a807-4240-893f-0cf2dda936cd service glance] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:07.982607 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1291/2589] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:33:12.585999 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13967481-3ec7-42b2-86c5-0a5a73aab365 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:12.586838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13967481-3ec7-42b2-86c5-0a5a73aab365 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:12.587607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13967481-3ec7-42b2-86c5-0a5a73aab365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:12.686814 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-13967481-3ec7-42b2-86c5-0a5a73aab365 None None] MFA Rules not processed for user `61ff29258a214817b8fc15b51fe8b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:12.691064 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-13967481-3ec7-42b2-86c5-0a5a73aab365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:12.770018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1299/2590] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Aug 7 19:33:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:12.787853 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17b24b-a807-4240-893f-0cf2dda936cd service glance] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:12.789408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-ef17b24b-a807-4240-893f-0cf2dda936cd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:12.879008 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17b24b-a807-4240-893f-0cf2dda936cd service glance] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:12.897088 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498ff25-b4b7-480d-8fef-f3bdc1ac8943 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db51971158a946c98a8ed716ecc5d72b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:12.898121 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498ff25-b4b7-480d-8fef-f3bdc1ac8943 service swift] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:12.898644 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498ff25-b4b7-480d-8fef-f3bdc1ac8943 service swift] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:12.898997 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498ff25-b4b7-480d-8fef-f3bdc1ac8943 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:12.909674 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498ff25-b4b7-480d-8fef-f3bdc1ac8943 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:12.911911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498ff25-b4b7-480d-8fef-f3bdc1ac8943 service swift] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:12.928210 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1292/2591] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Aug 7 19:33:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 19:33:15.356647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13967481-3ec7-42b2-86c5-0a5a73aab365 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:15.361206 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-13967481-3ec7-42b2-86c5-0a5a73aab365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:15.400107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498ff25-b4b7-480d-8fef-f3bdc1ac8943 service swift] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:15.401614 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b498ff25-b4b7-480d-8fef-f3bdc1ac8943 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:15.537489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:15.552371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498ff25-b4b7-480d-8fef-f3bdc1ac8943 service swift] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:15.565226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec26720-22f8-4b5c-80e8-2c4c0cda375c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:15.566313 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec26720-22f8-4b5c-80e8-2c4c0cda375c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:15.566767 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec26720-22f8-4b5c-80e8-2c4c0cda375c admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:15.567217 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec26720-22f8-4b5c-80e8-2c4c0cda375c admin admin] PATH_INFO: `/v3/users/1ea903d5ae1f4ad88f1fa12c29642370` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:15.569372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec26720-22f8-4b5c-80e8-2c4c0cda375c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ea903d5ae1f4ad88f1fa12c29642370)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:15.572486 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec26720-22f8-4b5c-80e8-2c4c0cda375c admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:15.582395 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13967481-3ec7-42b2-86c5-0a5a73aab365 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:15.593561 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe30258-72ce-4bf1-b42c-7a176a5a9a1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:15.594399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe30258-72ce-4bf1-b42c-7a176a5a9a1e service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:15.594776 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe30258-72ce-4bf1-b42c-7a176a5a9a1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:15.595141 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe30258-72ce-4bf1-b42c-7a176a5a9a1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:15.603740 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-afe30258-72ce-4bf1-b42c-7a176a5a9a1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:15.654960 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-9ec26720-22f8-4b5c-80e8-2c4c0cda375c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ea903d5ae1f4ad88f1fa12c29642370'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:15.667792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-9ec26720-22f8-4b5c-80e8-2c4c0cda375c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ea903d5ae1f4ad88f1fa12c29642370'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:15.688851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-9ec26720-22f8-4b5c-80e8-2c4c0cda375c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ea903d5ae1f4ad88f1fa12c29642370'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:15.690081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-9ec26720-22f8-4b5c-80e8-2c4c0cda375c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ea903d5ae1f4ad88f1fa12c29642370'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:15.703215 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-9ec26720-22f8-4b5c-80e8-2c4c0cda375c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ea903d5ae1f4ad88f1fa12c29642370'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:15.717206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1293/2592] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:33:15 2022] DELETE /identity/v3/users/1ea903d5ae1f4ad88f1fa12c29642370 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:33:15.719091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d1ade-ee88-4c3d-959f-df7240f27739 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:15.719532 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d1ade-ee88-4c3d-959f-df7240f27739 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:15.719809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d1ade-ee88-4c3d-959f-df7240f27739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:15.733715 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:15.750949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-4d1d1ade-ee88-4c3d-959f-df7240f27739 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:15.756593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4d1d1ade-ee88-4c3d-959f-df7240f27739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:15.817315 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe30258-72ce-4bf1-b42c-7a176a5a9a1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:15.819839 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe30258-72ce-4bf1-b42c-7a176a5a9a1e service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:15.846587 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1300/2593] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:33:15.899240 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1294/2594] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:15.905990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe30258-72ce-4bf1-b42c-7a176a5a9a1e service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:15.909996 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-afe30258-72ce-4bf1-b42c-7a176a5a9a1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:15.933397 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe30258-72ce-4bf1-b42c-7a176a5a9a1e service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:16.047153 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12b52d-af99-47d9-95c7-7d89fa0e0aa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:16.047822 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12b52d-af99-47d9-95c7-7d89fa0e0aa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:16.048262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12b52d-af99-47d9-95c7-7d89fa0e0aa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:16.048394 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12b52d-af99-47d9-95c7-7d89fa0e0aa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:16.059453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12b52d-af99-47d9-95c7-7d89fa0e0aa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:16.061045 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12b52d-af99-47d9-95c7-7d89fa0e0aa5 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:16.073042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1301/2595] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:33:16.200122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d1ade-ee88-4c3d-959f-df7240f27739 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:16.201951 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4d1d1ade-ee88-4c3d-959f-df7240f27739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:16.241962 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d1ade-ee88-4c3d-959f-df7240f27739 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:16.265838 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca5787-7914-4522-a3a8-0c3304746471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:16.267379 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca5787-7914-4522-a3a8-0c3304746471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:16.268058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca5787-7914-4522-a3a8-0c3304746471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:16.268571 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca5787-7914-4522-a3a8-0c3304746471 admin admin] PATH_INFO: `/v3/users/3412470ced8140fe982bb964be881b58` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:16.271430 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca5787-7914-4522-a3a8-0c3304746471 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3412470ced8140fe982bb964be881b58)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:16.284977 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca5787-7914-4522-a3a8-0c3304746471 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:16.405623 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e0ca5787-7914-4522-a3a8-0c3304746471 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3412470ced8140fe982bb964be881b58'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:16.428755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e0ca5787-7914-4522-a3a8-0c3304746471 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3412470ced8140fe982bb964be881b58'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:16.470968 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e0ca5787-7914-4522-a3a8-0c3304746471 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3412470ced8140fe982bb964be881b58'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:16.471838 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e0ca5787-7914-4522-a3a8-0c3304746471 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3412470ced8140fe982bb964be881b58'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:16.491758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-e0ca5787-7914-4522-a3a8-0c3304746471 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3412470ced8140fe982bb964be881b58'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:16.504351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1295/2596] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:33:16 2022] DELETE /identity/v3/users/3412470ced8140fe982bb964be881b58 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:16.510666 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa5206-a709-4344-9bdc-6cd69b5b18c9 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:16.511253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa5206-a709-4344-9bdc-6cd69b5b18c9 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:16.511620 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa5206-a709-4344-9bdc-6cd69b5b18c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:16.561400 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-4dfa5206-a709-4344-9bdc-6cd69b5b18c9 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:16.565100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4dfa5206-a709-4344-9bdc-6cd69b5b18c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:16.723317 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1302/2597] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:16.789967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca5787-7914-4522-a3a8-0c3304746471 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:16.792151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e0ca5787-7914-4522-a3a8-0c3304746471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:16.861690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa5206-a709-4344-9bdc-6cd69b5b18c9 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:16.863497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4dfa5206-a709-4344-9bdc-6cd69b5b18c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:16.891910 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa5206-a709-4344-9bdc-6cd69b5b18c9 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:16.953286 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:16.961464 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:16.961844 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:16.962991 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d admin admin] PATH_INFO: `/v3/projects/1f61c12cffca44f1a1cbef811b38b23b` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:16.964909 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f61c12cffca44f1a1cbef811b38b23b)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:16.978923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:16.992850 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca5787-7914-4522-a3a8-0c3304746471 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:17.003655 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb16eb-f90d-49ce-acdb-9d22400fa327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:17.004353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb16eb-f90d-49ce-acdb-9d22400fa327 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:17.004633 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb16eb-f90d-49ce-acdb-9d22400fa327 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:17.004901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb16eb-f90d-49ce-acdb-9d22400fa327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:17.030489 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb16eb-f90d-49ce-acdb-9d22400fa327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:17.032964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb16eb-f90d-49ce-acdb-9d22400fa327 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:17.059039 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1296/2598] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:33:17.105525 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:17.237485 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f61c12cffca44f1a1cbef811b38b23b', 'request_id': 'req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:17.263112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f61c12cffca44f1a1cbef811b38b23b', 'request_id': 'req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:17.289170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f61c12cffca44f1a1cbef811b38b23b', 'request_id': 'req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:17.321930 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d admin admin] The token cache is being invalidate because project 1f61c12cffca44f1a1cbef811b38b23b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:33:17.322213 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:17.324108 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:17.331396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1303/2599] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:33:16 2022] DELETE /identity/v3/projects/1f61c12cffca44f1a1cbef811b38b23b => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:17.337118 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb16eb-f90d-49ce-acdb-9d22400fa327 service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:17.352868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb16eb-f90d-49ce-acdb-9d22400fa327 service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:17.447982 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:17.594964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59e629-44d4-4f3a-87b7-f70b27243612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:17.597018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59e629-44d4-4f3a-87b7-f70b27243612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:17.597472 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59e629-44d4-4f3a-87b7-f70b27243612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:17.599245 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59e629-44d4-4f3a-87b7-f70b27243612 admin admin] PATH_INFO: `/v3/users/0f1faf903753454fb1b8015071ec7f2d` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:17.603954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59e629-44d4-4f3a-87b7-f70b27243612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f1faf903753454fb1b8015071ec7f2d)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:17.612446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59e629-44d4-4f3a-87b7-f70b27243612 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:17.727163 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6a59e629-44d4-4f3a-87b7-f70b27243612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f1faf903753454fb1b8015071ec7f2d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:17.742212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6a59e629-44d4-4f3a-87b7-f70b27243612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f1faf903753454fb1b8015071ec7f2d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:17.769165 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6a59e629-44d4-4f3a-87b7-f70b27243612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f1faf903753454fb1b8015071ec7f2d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:17.770113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6a59e629-44d4-4f3a-87b7-f70b27243612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f1faf903753454fb1b8015071ec7f2d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:17.782480 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6a59e629-44d4-4f3a-87b7-f70b27243612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f1faf903753454fb1b8015071ec7f2d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:17.799700 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1297/2600] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:33:17 2022] DELETE /identity/v3/users/0f1faf903753454fb1b8015071ec7f2d => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:18.537753 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:18.539344 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:18.584644 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251038d-7f29-4ba3-8d4c-98f4bea0e30d admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:18.831169 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740faa60-dc86-42e5-95aa-d45dd003f97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:18.833564 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740faa60-dc86-42e5-95aa-d45dd003f97e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:18.834092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740faa60-dc86-42e5-95aa-d45dd003f97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:18.836173 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740faa60-dc86-42e5-95aa-d45dd003f97e admin admin] PATH_INFO: `/v3/projects/12821f57ae714187b4b7c3955468fdb5` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:18.837949 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740faa60-dc86-42e5-95aa-d45dd003f97e admin admin] RBAC: Authorizing `identity:delete_project(project_id=12821f57ae714187b4b7c3955468fdb5)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:18.840585 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740faa60-dc86-42e5-95aa-d45dd003f97e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:18.922408 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-740faa60-dc86-42e5-95aa-d45dd003f97e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12821f57ae714187b4b7c3955468fdb5', 'request_id': 'req-740faa60-dc86-42e5-95aa-d45dd003f97e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:18.932025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-740faa60-dc86-42e5-95aa-d45dd003f97e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12821f57ae714187b4b7c3955468fdb5', 'request_id': 'req-740faa60-dc86-42e5-95aa-d45dd003f97e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:18.944432 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-740faa60-dc86-42e5-95aa-d45dd003f97e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12821f57ae714187b4b7c3955468fdb5', 'request_id': 'req-740faa60-dc86-42e5-95aa-d45dd003f97e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:18.969448 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-740faa60-dc86-42e5-95aa-d45dd003f97e admin admin] The token cache is being invalidate because project 12821f57ae714187b4b7c3955468fdb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:33:18.969730 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-740faa60-dc86-42e5-95aa-d45dd003f97e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:18.970410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-740faa60-dc86-42e5-95aa-d45dd003f97e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:18.972300 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1304/2601] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:33:18 2022] DELETE /identity/v3/projects/12821f57ae714187b4b7c3955468fdb5 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:18.978013 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac581d41-e7d8-4cce-9566-45c60a7a3035 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:18.978623 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac581d41-e7d8-4cce-9566-45c60a7a3035 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:18.978977 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac581d41-e7d8-4cce-9566-45c60a7a3035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:19.004140 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-ac581d41-e7d8-4cce-9566-45c60a7a3035 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:19.005792 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-ac581d41-e7d8-4cce-9566-45c60a7a3035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:19.082416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1298/2602] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:19.094174 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ca05b-d0db-44c2-8e43-9e2fbb1aceea None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:19.095568 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ca05b-d0db-44c2-8e43-9e2fbb1aceea None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:19.095568 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ca05b-d0db-44c2-8e43-9e2fbb1aceea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:19.135690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-115ca05b-d0db-44c2-8e43-9e2fbb1aceea None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:19.137096 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-115ca05b-d0db-44c2-8e43-9e2fbb1aceea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:19.147749 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1305/2603] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:19.152330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac581d41-e7d8-4cce-9566-45c60a7a3035 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:19.164834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac581d41-e7d8-4cce-9566-45c60a7a3035 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:19.170376 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f86e2e-e0a9-4c3b-a556-dabb341f446c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:19.171147 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f86e2e-e0a9-4c3b-a556-dabb341f446c admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:19.171489 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f86e2e-e0a9-4c3b-a556-dabb341f446c admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:19.171823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f86e2e-e0a9-4c3b-a556-dabb341f446c admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:19.172675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f86e2e-e0a9-4c3b-a556-dabb341f446c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:33:19.172969 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f86e2e-e0a9-4c3b-a556-dabb341f446c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:19.175112 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f86e2e-e0a9-4c3b-a556-dabb341f446c admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:19.189382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1299/2604] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:33:19.193913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ca05b-d0db-44c2-8e43-9e2fbb1aceea None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:19.204441 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ca05b-d0db-44c2-8e43-9e2fbb1aceea None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:19.209881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f792b0-c561-4409-b736-27091d54b93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:19.210530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f792b0-c561-4409-b736-27091d54b93d admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:19.210809 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f792b0-c561-4409-b736-27091d54b93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:19.211079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f792b0-c561-4409-b736-27091d54b93d admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:19.211750 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f792b0-c561-4409-b736-27091d54b93d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:19.213905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f792b0-c561-4409-b736-27091d54b93d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:19.277729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1306/2605] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:33:19 2022] POST /identity/v3/projects => generated 377 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:19.281910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f86e2e-e0a9-4c3b-a556-dabb341f446c admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:19.293420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f86e2e-e0a9-4c3b-a556-dabb341f446c admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:19.337645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:19.404958 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f792b0-c561-4409-b736-27091d54b93d admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:19.406859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-64f792b0-c561-4409-b736-27091d54b93d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:19.409168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e5d05-63b2-4635-981f-0842b3371645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:19.410151 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e5d05-63b2-4635-981f-0842b3371645 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:19.410602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e5d05-63b2-4635-981f-0842b3371645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:19.411032 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e5d05-63b2-4635-981f-0842b3371645 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:19.411820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e5d05-63b2-4635-981f-0842b3371645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:19.414800 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e5d05-63b2-4635-981f-0842b3371645 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:19.430495 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f792b0-c561-4409-b736-27091d54b93d admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:19.440263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c72c46-978d-4be8-bcb3-3912dec7fe86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:19.440860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c72c46-978d-4be8-bcb3-3912dec7fe86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:19.441152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c72c46-978d-4be8-bcb3-3912dec7fe86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:19.441429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c72c46-978d-4be8-bcb3-3912dec7fe86 admin admin] PATH_INFO: `/v3/projects/27cf1027dd8b4747b707ba9269d29a47` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:19.444480 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c72c46-978d-4be8-bcb3-3912dec7fe86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27cf1027dd8b4747b707ba9269d29a47)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:19.446475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c72c46-978d-4be8-bcb3-3912dec7fe86 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:19.448629 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1300/2606] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:33:19 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:19.453068 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e5d05-63b2-4635-981f-0842b3371645 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:19.482500 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e5d05-63b2-4635-981f-0842b3371645 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:19.496778 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:19.521885 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c2562-6477-41eb-bad6-2d22f18bd20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:19.522550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c2562-6477-41eb-bad6-2d22f18bd20c admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:19.522900 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c2562-6477-41eb-bad6-2d22f18bd20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:19.523172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c2562-6477-41eb-bad6-2d22f18bd20c admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:19.523833 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c2562-6477-41eb-bad6-2d22f18bd20c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:19.525560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392c2562-6477-41eb-bad6-2d22f18bd20c admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:19.544132 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1301/2607] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:33:19.551426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c2562-6477-41eb-bad6-2d22f18bd20c admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:19.560518 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-78c72c46-978d-4be8-bcb3-3912dec7fe86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27cf1027dd8b4747b707ba9269d29a47', 'request_id': 'req-78c72c46-978d-4be8-bcb3-3912dec7fe86'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:19.568217 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-78c72c46-978d-4be8-bcb3-3912dec7fe86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27cf1027dd8b4747b707ba9269d29a47', 'request_id': 'req-78c72c46-978d-4be8-bcb3-3912dec7fe86'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:19.572705 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392c2562-6477-41eb-bad6-2d22f18bd20c admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:19.591136 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-78c72c46-978d-4be8-bcb3-3912dec7fe86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27cf1027dd8b4747b707ba9269d29a47', 'request_id': 'req-78c72c46-978d-4be8-bcb3-3912dec7fe86'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:19.637620 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-78c72c46-978d-4be8-bcb3-3912dec7fe86 admin admin] The token cache is being invalidate because project 27cf1027dd8b4747b707ba9269d29a47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:33:19.638417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-78c72c46-978d-4be8-bcb3-3912dec7fe86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:19.639653 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-78c72c46-978d-4be8-bcb3-3912dec7fe86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:19.645606 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1307/2608] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:33:19 2022] DELETE /identity/v3/projects/27cf1027dd8b4747b707ba9269d29a47 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:33:19.657854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9721f-1675-48a3-ae51-44d65e30f996 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:19.658283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9721f-1675-48a3-ae51-44d65e30f996 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:19.658677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc9721f-1675-48a3-ae51-44d65e30f996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:19.693658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-1bc9721f-1675-48a3-ae51-44d65e30f996 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:19.695788 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1bc9721f-1675-48a3-ae51-44d65e30f996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:19.742382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285fe7b-a7b8-44a5-884b-c96746cbc5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:19.743501 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285fe7b-a7b8-44a5-884b-c96746cbc5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:19.744023 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285fe7b-a7b8-44a5-884b-c96746cbc5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:19.744409 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285fe7b-a7b8-44a5-884b-c96746cbc5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:19.745191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285fe7b-a7b8-44a5-884b-c96746cbc5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:19.749022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1285fe7b-a7b8-44a5-884b-c96746cbc5a4 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:19.769910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1302/2609] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:19.773084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285fe7b-a7b8-44a5-884b-c96746cbc5a4 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:19.775163 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-1285fe7b-a7b8-44a5-884b-c96746cbc5a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:19.791427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285fe7b-a7b8-44a5-884b-c96746cbc5a4 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:19.798514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6be060-9fea-4147-99b5-9573f358ca9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:19.799632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6be060-9fea-4147-99b5-9573f358ca9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:19.800119 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6be060-9fea-4147-99b5-9573f358ca9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:19.800641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6be060-9fea-4147-99b5-9573f358ca9b admin admin] PATH_INFO: `/v3/projects/bdc0726934c64051b49ed81a2abc5c89/users/656a586e718543bfb095790441c28fe8/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:19.824079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6be060-9fea-4147-99b5-9573f358ca9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc0726934c64051b49ed81a2abc5c89, user_id=656a586e718543bfb095790441c28fe8, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:19.827748 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6be060-9fea-4147-99b5-9573f358ca9b admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:19.912343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1303/2610] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:19 2022] PUT /identity/v3/projects/bdc0726934c64051b49ed81a2abc5c89/users/656a586e718543bfb095790441c28fe8/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:19.912343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075b844-957c-4386-89a5-32ddb1da3d93 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:19.912343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075b844-957c-4386-89a5-32ddb1da3d93 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:19.912343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075b844-957c-4386-89a5-32ddb1da3d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:19.913389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1308/2611] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:19.913485 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-4075b844-957c-4386-89a5-32ddb1da3d93 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:19.913485 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4075b844-957c-4386-89a5-32ddb1da3d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:19.914275 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347fb5a-3a65-4356-8bcd-2bef0c23de37 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:19.914496 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347fb5a-3a65-4356-8bcd-2bef0c23de37 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:19.914766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e347fb5a-3a65-4356-8bcd-2bef0c23de37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:20.001181 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-e347fb5a-3a65-4356-8bcd-2bef0c23de37 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:20.003352 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e347fb5a-3a65-4356-8bcd-2bef0c23de37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:20.101050 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1304/2612] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:20.113907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075b844-957c-4386-89a5-32ddb1da3d93 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:20.117677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4075b844-957c-4386-89a5-32ddb1da3d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:20.152293 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:20.195420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1309/2613] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:20.198093 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347fb5a-3a65-4356-8bcd-2bef0c23de37 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:20.210620 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e347fb5a-3a65-4356-8bcd-2bef0c23de37 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:20.217501 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644ebe2-e285-4d96-b1a1-db9e0df0bec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:20.218325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644ebe2-e285-4d96-b1a1-db9e0df0bec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:20.218607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644ebe2-e285-4d96-b1a1-db9e0df0bec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:20.218887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644ebe2-e285-4d96-b1a1-db9e0df0bec5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:20.219950 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644ebe2-e285-4d96-b1a1-db9e0df0bec5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:33:20.220682 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644ebe2-e285-4d96-b1a1-db9e0df0bec5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:20.222409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644ebe2-e285-4d96-b1a1-db9e0df0bec5 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:20.241634 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1310/2614] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:33:20.251081 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644ebe2-e285-4d96-b1a1-db9e0df0bec5 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:20.251905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075b844-957c-4386-89a5-32ddb1da3d93 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:20.265310 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5b86c-ae7e-47f3-b18e-ae35851177db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:20.266527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5b86c-ae7e-47f3-b18e-ae35851177db service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:20.267017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5b86c-ae7e-47f3-b18e-ae35851177db service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:20.267473 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a5b86c-ae7e-47f3-b18e-ae35851177db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:20.268850 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644ebe2-e285-4d96-b1a1-db9e0df0bec5 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:20.276838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3914c-bfe9-452f-a0bd-2d7dd4c6948b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:20.277640 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3914c-bfe9-452f-a0bd-2d7dd4c6948b admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:20.277918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3914c-bfe9-452f-a0bd-2d7dd4c6948b admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:20.278303 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad3914c-bfe9-452f-a0bd-2d7dd4c6948b admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:20.279100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3914c-bfe9-452f-a0bd-2d7dd4c6948b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:20.281664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad3914c-bfe9-452f-a0bd-2d7dd4c6948b admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:20.289782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5b86c-ae7e-47f3-b18e-ae35851177db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:20.292415 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a5b86c-ae7e-47f3-b18e-ae35851177db service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:20.315014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1305/2615] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:33:20.333330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1311/2616] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:33:20 2022] POST /identity/v3/projects => generated 367 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:20.334725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5b86c-ae7e-47f3-b18e-ae35851177db service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:20.345644 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a5b86c-ae7e-47f3-b18e-ae35851177db service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:20.475960 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c9f30-84d3-4c78-9c71-87dbd1c5dbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:20.476659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c9f30-84d3-4c78-9c71-87dbd1c5dbd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:20.476990 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c9f30-84d3-4c78-9c71-87dbd1c5dbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:20.478618 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c9f30-84d3-4c78-9c71-87dbd1c5dbd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:20.480588 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c9f30-84d3-4c78-9c71-87dbd1c5dbd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:20.486073 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c9f30-84d3-4c78-9c71-87dbd1c5dbd7 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:20.587563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1306/2617] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:33:20 2022] POST /identity/v3/users => generated 284 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:20.591925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3914c-bfe9-452f-a0bd-2d7dd4c6948b admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:20.616646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad3914c-bfe9-452f-a0bd-2d7dd4c6948b admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:20.632589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d738a-a282-4474-8fa7-3c90b7e8d7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:20.635767 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d738a-a282-4474-8fa7-3c90b7e8d7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:20.636199 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d738a-a282-4474-8fa7-3c90b7e8d7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:20.636681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d738a-a282-4474-8fa7-3c90b7e8d7e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:20.637550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d738a-a282-4474-8fa7-3c90b7e8d7e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:20.645579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d738a-a282-4474-8fa7-3c90b7e8d7e4 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:20.673803 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1312/2618] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:33:20.674795 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c9f30-84d3-4c78-9c71-87dbd1c5dbd7 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:20.690623 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c9f30-84d3-4c78-9c71-87dbd1c5dbd7 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:20.703574 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b7ecb-2d07-4994-ae6f-53dd82967e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:20.704411 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b7ecb-2d07-4994-ae6f-53dd82967e27 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:20.704766 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b7ecb-2d07-4994-ae6f-53dd82967e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:20.705037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b7ecb-2d07-4994-ae6f-53dd82967e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:20.705698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b7ecb-2d07-4994-ae6f-53dd82967e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:20.707397 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30b7ecb-2d07-4994-ae6f-53dd82967e27 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:20.757663 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1307/2619] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:33:20.762379 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d738a-a282-4474-8fa7-3c90b7e8d7e4 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:20.774330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:20.780987 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d738a-a282-4474-8fa7-3c90b7e8d7e4 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:20.798410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a837b-5dd3-4caa-83f8-2c6ecac04b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:20.799617 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a837b-5dd3-4caa-83f8-2c6ecac04b3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:20.800090 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a837b-5dd3-4caa-83f8-2c6ecac04b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:20.800657 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a837b-5dd3-4caa-83f8-2c6ecac04b3f admin admin] PATH_INFO: `/v3/projects/81cc7679f646447b8cae051454fa93dd/users/d17060a689774af18b1d5806c82dfd07/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:20.841907 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a837b-5dd3-4caa-83f8-2c6ecac04b3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=81cc7679f646447b8cae051454fa93dd, user_id=d17060a689774af18b1d5806c82dfd07, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:20.846157 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a837b-5dd3-4caa-83f8-2c6ecac04b3f admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:20.886521 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1313/2620] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:20 2022] PUT /identity/v3/projects/81cc7679f646447b8cae051454fa93dd/users/d17060a689774af18b1d5806c82dfd07/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:20.888403 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b8ae4-39c7-44a1-8b97-c79828ce0b9b None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:20.888881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b8ae4-39c7-44a1-8b97-c79828ce0b9b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:20.889536 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b8ae4-39c7-44a1-8b97-c79828ce0b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:20.950775 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-527b8ae4-39c7-44a1-8b97-c79828ce0b9b None None] MFA Rules not processed for user `d17060a689774af18b1d5806c82dfd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:20.952066 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-527b8ae4-39c7-44a1-8b97-c79828ce0b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:21.103175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1308/2621] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:21.107656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a837b-5dd3-4caa-83f8-2c6ecac04b3f admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:21.300660 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a837b-5dd3-4caa-83f8-2c6ecac04b3f admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:21.307576 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36160a-7190-4fc8-8df8-b0065369f467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:21.309252 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36160a-7190-4fc8-8df8-b0065369f467 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:21.309828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36160a-7190-4fc8-8df8-b0065369f467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:21.310330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36160a-7190-4fc8-8df8-b0065369f467 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:21.311555 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36160a-7190-4fc8-8df8-b0065369f467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:21.314694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36160a-7190-4fc8-8df8-b0065369f467 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:21.402230 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1314/2622] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:33:21 2022] POST /identity/v3/projects => generated 367 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:21.403546 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b8ae4-39c7-44a1-8b97-c79828ce0b9b None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:21.428101 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b8ae4-39c7-44a1-8b97-c79828ce0b9b None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:21.512734 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:21.593081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c9326-7774-4c52-9022-4d3e366a5443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:21.593781 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c9326-7774-4c52-9022-4d3e366a5443 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:21.594128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c9326-7774-4c52-9022-4d3e366a5443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:21.594592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c9326-7774-4c52-9022-4d3e366a5443 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:21.595398 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c9326-7774-4c52-9022-4d3e366a5443 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:21.597695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c9326-7774-4c52-9022-4d3e366a5443 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:21.643861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1309/2623] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:33:21 2022] POST /identity/v3/users => generated 290 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:21.650227 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36160a-7190-4fc8-8df8-b0065369f467 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:21.682362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36160a-7190-4fc8-8df8-b0065369f467 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:21.691898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39499a30-9ca5-4739-9367-aed45f40a5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:21.692581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39499a30-9ca5-4739-9367-aed45f40a5bf admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:21.692870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39499a30-9ca5-4739-9367-aed45f40a5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:21.693149 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39499a30-9ca5-4739-9367-aed45f40a5bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:21.693804 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39499a30-9ca5-4739-9367-aed45f40a5bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:21.699467 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39499a30-9ca5-4739-9367-aed45f40a5bf admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:21.728568 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1315/2624] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:21.734673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c9326-7774-4c52-9022-4d3e366a5443 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:21.751161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c9326-7774-4c52-9022-4d3e366a5443 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:21.766598 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d5266-f177-4164-98ec-7b089ba59f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:21.768647 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d5266-f177-4164-98ec-7b089ba59f7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:21.770195 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d5266-f177-4164-98ec-7b089ba59f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:21.770745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d5266-f177-4164-98ec-7b089ba59f7a admin admin] PATH_INFO: `/v3/projects/c190d29b390a4bf4a1153e756548ec45/users/2f09b7e692784db99ac82345e554b624/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:21.814063 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d5266-f177-4164-98ec-7b089ba59f7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c190d29b390a4bf4a1153e756548ec45, user_id=2f09b7e692784db99ac82345e554b624, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:21.818077 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d5266-f177-4164-98ec-7b089ba59f7a admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:21.850374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1310/2625] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:21 2022] PUT /identity/v3/projects/c190d29b390a4bf4a1153e756548ec45/users/2f09b7e692784db99ac82345e554b624/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:21.854588 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39499a30-9ca5-4739-9367-aed45f40a5bf admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:21.873918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39499a30-9ca5-4739-9367-aed45f40a5bf admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:22.027587 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eea8402-3aa4-4b37-b986-d2908355ada4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:22.028249 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea8402-3aa4-4b37-b986-d2908355ada4 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:22.028526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea8402-3aa4-4b37-b986-d2908355ada4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:22.028797 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea8402-3aa4-4b37-b986-d2908355ada4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:22.029457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea8402-3aa4-4b37-b986-d2908355ada4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:22.036025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eea8402-3aa4-4b37-b986-d2908355ada4 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:22.055816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1316/2626] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:22.057428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d5266-f177-4164-98ec-7b089ba59f7a admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:22.072467 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d5266-f177-4164-98ec-7b089ba59f7a admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:22.105985 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836357e-2e12-4c8d-b367-79ef2d71969e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:22.106772 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836357e-2e12-4c8d-b367-79ef2d71969e admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:22.106966 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836357e-2e12-4c8d-b367-79ef2d71969e admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:22.107390 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836357e-2e12-4c8d-b367-79ef2d71969e admin admin] PATH_INFO: `/v3/projects/c190d29b390a4bf4a1153e756548ec45/users/2f09b7e692784db99ac82345e554b624/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:22.116356 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836357e-2e12-4c8d-b367-79ef2d71969e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c190d29b390a4bf4a1153e756548ec45, user_id=2f09b7e692784db99ac82345e554b624, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:22.126912 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836357e-2e12-4c8d-b367-79ef2d71969e admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:22.167159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1311/2627] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:22 2022] PUT /identity/v3/projects/c190d29b390a4bf4a1153e756548ec45/users/2f09b7e692784db99ac82345e554b624/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:22.178856 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f952a7-f947-431b-a806-2f5531968fe4 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:22.179539 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f952a7-f947-431b-a806-2f5531968fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:22.180204 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f952a7-f947-431b-a806-2f5531968fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:22.279100 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-46f952a7-f947-431b-a806-2f5531968fe4 None None] MFA Rules not processed for user `2f09b7e692784db99ac82345e554b624`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:22.283117 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-46f952a7-f947-431b-a806-2f5531968fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:22.447021 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:22.557428 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1317/2628] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:22.566837 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836357e-2e12-4c8d-b367-79ef2d71969e admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:22.568853 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-6836357e-2e12-4c8d-b367-79ef2d71969e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:22.812886 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836357e-2e12-4c8d-b367-79ef2d71969e admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:22.826354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7001fb3-008b-47de-99f9-2b05a9d7d5e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:22.828041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7001fb3-008b-47de-99f9-2b05a9d7d5e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:22.840439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7001fb3-008b-47de-99f9-2b05a9d7d5e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:22.840724 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7001fb3-008b-47de-99f9-2b05a9d7d5e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:22.856113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7001fb3-008b-47de-99f9-2b05a9d7d5e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:22.860036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7001fb3-008b-47de-99f9-2b05a9d7d5e6 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:22.920939 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1312/2629] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:33:23.084204 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f952a7-f947-431b-a806-2f5531968fe4 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:23.092504 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-46f952a7-f947-431b-a806-2f5531968fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:23.116851 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f952a7-f947-431b-a806-2f5531968fe4 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:23.124726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9d8e9-806d-46eb-a2f4-342b2612f1f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:23.125485 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9d8e9-806d-46eb-a2f4-342b2612f1f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:23.125847 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9d8e9-806d-46eb-a2f4-342b2612f1f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:23.126155 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9d8e9-806d-46eb-a2f4-342b2612f1f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:23.284474 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9d8e9-806d-46eb-a2f4-342b2612f1f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:23.286017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9d8e9-806d-46eb-a2f4-342b2612f1f6 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:23.317380 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1318/2630] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:33:26.930884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7001fb3-008b-47de-99f9-2b05a9d7d5e6 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:26.934337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c7001fb3-008b-47de-99f9-2b05a9d7d5e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:27.185370 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7001fb3-008b-47de-99f9-2b05a9d7d5e6 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:27.211211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2e68e7-c1f1-47b2-bba5-b41cbdefc0d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:27.212261 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e68e7-c1f1-47b2-bba5-b41cbdefc0d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:27.212737 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e68e7-c1f1-47b2-bba5-b41cbdefc0d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:27.213223 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2e68e7-c1f1-47b2-bba5-b41cbdefc0d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:27.218000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4d2e68e7-c1f1-47b2-bba5-b41cbdefc0d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:27.342262 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:27.405778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e68e7-c1f1-47b2-bba5-b41cbdefc0d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:27.407424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2e68e7-c1f1-47b2-bba5-b41cbdefc0d0 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:27.435461 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1313/2631] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:33:29.547209 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9d8e9-806d-46eb-a2f4-342b2612f1f6 service cinder] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:29.575634 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9d8e9-806d-46eb-a2f4-342b2612f1f6 service cinder] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:29.599404 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40ce37-cc22-43b6-b13f-5ca8e2ec2d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:29.601984 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40ce37-cc22-43b6-b13f-5ca8e2ec2d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:29.602839 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40ce37-cc22-43b6-b13f-5ca8e2ec2d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:29.604990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40ce37-cc22-43b6-b13f-5ca8e2ec2d2c admin admin] PATH_INFO: `/v3/users/d17060a689774af18b1d5806c82dfd07` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:29.611729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40ce37-cc22-43b6-b13f-5ca8e2ec2d2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d17060a689774af18b1d5806c82dfd07)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:29.616762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f40ce37-cc22-43b6-b13f-5ca8e2ec2d2c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:29.626546 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb3de4-f4df-4d84-8b54-eba695be4898 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:29.627409 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb3de4-f4df-4d84-8b54-eba695be4898 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:29.627409 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb3de4-f4df-4d84-8b54-eba695be4898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:29.664385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-63eb3de4-f4df-4d84-8b54-eba695be4898 None None] MFA Rules not processed for user `61ff29258a214817b8fc15b51fe8b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:29.665761 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-63eb3de4-f4df-4d84-8b54-eba695be4898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:29.747171 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-8f40ce37-cc22-43b6-b13f-5ca8e2ec2d2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd17060a689774af18b1d5806c82dfd07'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:29.780977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-8f40ce37-cc22-43b6-b13f-5ca8e2ec2d2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd17060a689774af18b1d5806c82dfd07'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:29.799778 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-8f40ce37-cc22-43b6-b13f-5ca8e2ec2d2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd17060a689774af18b1d5806c82dfd07'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:29.801819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-8f40ce37-cc22-43b6-b13f-5ca8e2ec2d2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd17060a689774af18b1d5806c82dfd07'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:29.803416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-8f40ce37-cc22-43b6-b13f-5ca8e2ec2d2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd17060a689774af18b1d5806c82dfd07'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:29.851706 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1319/2632] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:33:29 2022] DELETE /identity/v3/users/d17060a689774af18b1d5806c82dfd07 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:29.859679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c368d7e-c24d-49a8-b21c-625981f8bdb0 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:29.861925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c368d7e-c24d-49a8-b21c-625981f8bdb0 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:29.862351 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c368d7e-c24d-49a8-b21c-625981f8bdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:29.871848 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1314/2633] 10.222.0.17 () {58 vars in 967 bytes} [Sun Aug 7 19:33:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:29.893805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb3de4-f4df-4d84-8b54-eba695be4898 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:29.901425 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-63eb3de4-f4df-4d84-8b54-eba695be4898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:29.912364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-2c368d7e-c24d-49a8-b21c-625981f8bdb0 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:29.913902 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-2c368d7e-c24d-49a8-b21c-625981f8bdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:29.975457 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:30.028211 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1320/2634] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:30.060043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c368d7e-c24d-49a8-b21c-625981f8bdb0 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:30.061594 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-2c368d7e-c24d-49a8-b21c-625981f8bdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:30.115206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb3de4-f4df-4d84-8b54-eba695be4898 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:30.200312 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c368d7e-c24d-49a8-b21c-625981f8bdb0 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:30.234289 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55468c-451d-4335-9591-0223b8038d2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:30.239846 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55468c-451d-4335-9591-0223b8038d2b service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:30.243433 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55468c-451d-4335-9591-0223b8038d2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:30.248161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55468c-451d-4335-9591-0223b8038d2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:30.284519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55468c-451d-4335-9591-0223b8038d2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:30.286036 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55468c-451d-4335-9591-0223b8038d2b service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:30.317078 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1321/2635] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:33:30.329620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa03b5-cb6e-41f9-bf5c-06a4ae8e17bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db51971158a946c98a8ed716ecc5d72b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:30.330283 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa03b5-cb6e-41f9-bf5c-06a4ae8e17bf service swift] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:30.330719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa03b5-cb6e-41f9-bf5c-06a4ae8e17bf service swift] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:30.330995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa03b5-cb6e-41f9-bf5c-06a4ae8e17bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:30.396958 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:30.476428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa03b5-cb6e-41f9-bf5c-06a4ae8e17bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:30.479051 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa03b5-cb6e-41f9-bf5c-06a4ae8e17bf service swift] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:30.507486 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1315/2636] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Aug 7 19:33:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 19:33:31.529234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55468c-451d-4335-9591-0223b8038d2b service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:31.532936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5d55468c-451d-4335-9591-0223b8038d2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:31.604479 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55468c-451d-4335-9591-0223b8038d2b service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:31.616896 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03479a2-c4be-47a2-9d88-5286f897d447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:31.618510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03479a2-c4be-47a2-9d88-5286f897d447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:31.619241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03479a2-c4be-47a2-9d88-5286f897d447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:31.621867 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03479a2-c4be-47a2-9d88-5286f897d447 admin admin] PATH_INFO: `/v3/projects/81cc7679f646447b8cae051454fa93dd` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:31.628663 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03479a2-c4be-47a2-9d88-5286f897d447 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81cc7679f646447b8cae051454fa93dd)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:31.638007 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03479a2-c4be-47a2-9d88-5286f897d447 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:31.869833 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e03479a2-c4be-47a2-9d88-5286f897d447 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81cc7679f646447b8cae051454fa93dd', 'request_id': 'req-e03479a2-c4be-47a2-9d88-5286f897d447'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:31.908874 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e03479a2-c4be-47a2-9d88-5286f897d447 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81cc7679f646447b8cae051454fa93dd', 'request_id': 'req-e03479a2-c4be-47a2-9d88-5286f897d447'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:31.947212 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e03479a2-c4be-47a2-9d88-5286f897d447 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81cc7679f646447b8cae051454fa93dd', 'request_id': 'req-e03479a2-c4be-47a2-9d88-5286f897d447'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:32.055742 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e03479a2-c4be-47a2-9d88-5286f897d447 admin admin] The token cache is being invalidate because project 81cc7679f646447b8cae051454fa93dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:33:32.056292 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e03479a2-c4be-47a2-9d88-5286f897d447 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:32.075698 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e03479a2-c4be-47a2-9d88-5286f897d447 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:32.079373 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1322/2637] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:33:31 2022] DELETE /identity/v3/projects/81cc7679f646447b8cae051454fa93dd => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:32.083188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa03b5-cb6e-41f9-bf5c-06a4ae8e17bf service swift] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:32.106606 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa03b5-cb6e-41f9-bf5c-06a4ae8e17bf service swift] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:32.224053 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbbcaf-1e52-4d1b-8d3d-ebca9e0719fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:32.224969 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbbcaf-1e52-4d1b-8d3d-ebca9e0719fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:32.224969 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbbcaf-1e52-4d1b-8d3d-ebca9e0719fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:32.225455 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbbcaf-1e52-4d1b-8d3d-ebca9e0719fd admin admin] PATH_INFO: `/v3/users/2f09b7e692784db99ac82345e554b624` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:32.226934 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbbcaf-1e52-4d1b-8d3d-ebca9e0719fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f09b7e692784db99ac82345e554b624)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:32.229300 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbbcaf-1e52-4d1b-8d3d-ebca9e0719fd admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:32.344376 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-1bcbbcaf-1e52-4d1b-8d3d-ebca9e0719fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f09b7e692784db99ac82345e554b624'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:32.372298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-1bcbbcaf-1e52-4d1b-8d3d-ebca9e0719fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f09b7e692784db99ac82345e554b624'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:32.383178 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446384d7-d00b-4f9a-9847-ea2ea018725b None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:32.383782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446384d7-d00b-4f9a-9847-ea2ea018725b None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:32.384316 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446384d7-d00b-4f9a-9847-ea2ea018725b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:32.396746 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-1bcbbcaf-1e52-4d1b-8d3d-ebca9e0719fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f09b7e692784db99ac82345e554b624'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:32.397750 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-1bcbbcaf-1e52-4d1b-8d3d-ebca9e0719fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f09b7e692784db99ac82345e554b624'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:32.418937 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-1bcbbcaf-1e52-4d1b-8d3d-ebca9e0719fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f09b7e692784db99ac82345e554b624'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:32.435850 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1316/2638] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:33:32 2022] DELETE /identity/v3/users/2f09b7e692784db99ac82345e554b624 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:32.528932 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-446384d7-d00b-4f9a-9847-ea2ea018725b None None] MFA Rules not processed for user `656a586e718543bfb095790441c28fe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:32.530557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-446384d7-d00b-4f9a-9847-ea2ea018725b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:32.680223 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1323/2639] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:32.680925 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbbcaf-1e52-4d1b-8d3d-ebca9e0719fd admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:32.682449 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-1bcbbcaf-1e52-4d1b-8d3d-ebca9e0719fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:32.760089 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:32.830019 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbbcaf-1e52-4d1b-8d3d-ebca9e0719fd admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:32.840984 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86895bb-c1cd-4782-a008-a8612d6823f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:32.845338 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86895bb-c1cd-4782-a008-a8612d6823f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:32.845904 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86895bb-c1cd-4782-a008-a8612d6823f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:32.846464 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86895bb-c1cd-4782-a008-a8612d6823f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:32.848105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86895bb-c1cd-4782-a008-a8612d6823f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:32.852688 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86895bb-c1cd-4782-a008-a8612d6823f2 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:32.930398 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1317/2640] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:33:32 2022] POST /identity/v3/projects => generated 379 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:33:32.934138 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446384d7-d00b-4f9a-9847-ea2ea018725b None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:32.941815 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:32.950887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446384d7-d00b-4f9a-9847-ea2ea018725b None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:33.103718 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790a91c-8c18-48ba-9b4f-10b1bd7d55b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:33.104520 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790a91c-8c18-48ba-9b4f-10b1bd7d55b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:33.104809 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790a91c-8c18-48ba-9b4f-10b1bd7d55b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:33.105084 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9790a91c-8c18-48ba-9b4f-10b1bd7d55b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:33.105870 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790a91c-8c18-48ba-9b4f-10b1bd7d55b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:33.109343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9790a91c-8c18-48ba-9b4f-10b1bd7d55b3 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:33.161251 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86895bb-c1cd-4782-a008-a8612d6823f2 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:33.170981 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1324/2641] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:33:32 2022] POST /identity/v3/users => generated 296 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:33.179327 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86895bb-c1cd-4782-a008-a8612d6823f2 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:33.188996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee5532-2ed3-416d-bb78-da3b9181696b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:33.189832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee5532-2ed3-416d-bb78-da3b9181696b admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:33.190214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee5532-2ed3-416d-bb78-da3b9181696b admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:33.190517 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee5532-2ed3-416d-bb78-da3b9181696b admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:33.192118 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee5532-2ed3-416d-bb78-da3b9181696b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:33.195880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee5532-2ed3-416d-bb78-da3b9181696b admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:33.220546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790a91c-8c18-48ba-9b4f-10b1bd7d55b3 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:33.221581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1318/2642] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:33.240178 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9790a91c-8c18-48ba-9b4f-10b1bd7d55b3 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:33.248578 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4cdf7-58b0-4109-86d6-4b3a6d7145c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:33.249305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4cdf7-58b0-4109-86d6-4b3a6d7145c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:33.249594 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4cdf7-58b0-4109-86d6-4b3a6d7145c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:33.249937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4cdf7-58b0-4109-86d6-4b3a6d7145c0 admin admin] PATH_INFO: `/v3/projects/ac61655af79e4fcf9c54979977957d2e/users/e58ea4f0b3414240ab7eb521e7b92db1/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:33.314354 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4cdf7-58b0-4109-86d6-4b3a6d7145c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac61655af79e4fcf9c54979977957d2e, user_id=e58ea4f0b3414240ab7eb521e7b92db1, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:33.322309 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db4cdf7-58b0-4109-86d6-4b3a6d7145c0 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:33.352050 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1325/2643] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:33 2022] PUT /identity/v3/projects/ac61655af79e4fcf9c54979977957d2e/users/e58ea4f0b3414240ab7eb521e7b92db1/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:33.358041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee5532-2ed3-416d-bb78-da3b9181696b admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:33.373722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee5532-2ed3-416d-bb78-da3b9181696b admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:33.520808 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dead895-6e33-4bdc-9d71-d67953639181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:33.521751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dead895-6e33-4bdc-9d71-d67953639181 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:33.522127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dead895-6e33-4bdc-9d71-d67953639181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:33.522505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dead895-6e33-4bdc-9d71-d67953639181 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:33.524806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dead895-6e33-4bdc-9d71-d67953639181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:33.530351 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dead895-6e33-4bdc-9d71-d67953639181 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:33.557855 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1319/2644] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:33.562654 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4cdf7-58b0-4109-86d6-4b3a6d7145c0 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:33.581139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dead895-6e33-4bdc-9d71-d67953639181 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:33.584265 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4dead895-6e33-4bdc-9d71-d67953639181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:33.588683 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db4cdf7-58b0-4109-86d6-4b3a6d7145c0 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:33.617069 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f35a8-db50-4632-aa3b-c3be453b02d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:33.619478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f35a8-db50-4632-aa3b-c3be453b02d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:33.620095 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f35a8-db50-4632-aa3b-c3be453b02d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:33.620646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f35a8-db50-4632-aa3b-c3be453b02d6 admin admin] PATH_INFO: `/v3/projects/ac61655af79e4fcf9c54979977957d2e/users/e58ea4f0b3414240ab7eb521e7b92db1/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:33.623714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f35a8-db50-4632-aa3b-c3be453b02d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac61655af79e4fcf9c54979977957d2e, user_id=e58ea4f0b3414240ab7eb521e7b92db1, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:33.632225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f35a8-db50-4632-aa3b-c3be453b02d6 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:33.638595 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dead895-6e33-4bdc-9d71-d67953639181 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:33.650358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffaba4-447b-4619-9928-8ed9aed5ca16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:33.651039 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffaba4-447b-4619-9928-8ed9aed5ca16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:33.651405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffaba4-447b-4619-9928-8ed9aed5ca16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:33.652019 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffaba4-447b-4619-9928-8ed9aed5ca16 admin admin] PATH_INFO: `/v3/projects/c190d29b390a4bf4a1153e756548ec45` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:33.653695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffaba4-447b-4619-9928-8ed9aed5ca16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c190d29b390a4bf4a1153e756548ec45)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:33.655707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffaba4-447b-4619-9928-8ed9aed5ca16 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:33.694730 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1326/2645] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:33 2022] PUT /identity/v3/projects/ac61655af79e4fcf9c54979977957d2e/users/e58ea4f0b3414240ab7eb521e7b92db1/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:33:33.816821 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7cffaba4-447b-4619-9928-8ed9aed5ca16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c190d29b390a4bf4a1153e756548ec45', 'request_id': 'req-7cffaba4-447b-4619-9928-8ed9aed5ca16'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:33.829316 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7cffaba4-447b-4619-9928-8ed9aed5ca16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c190d29b390a4bf4a1153e756548ec45', 'request_id': 'req-7cffaba4-447b-4619-9928-8ed9aed5ca16'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:33.840534 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7cffaba4-447b-4619-9928-8ed9aed5ca16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c190d29b390a4bf4a1153e756548ec45', 'request_id': 'req-7cffaba4-447b-4619-9928-8ed9aed5ca16'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:33.872765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7cffaba4-447b-4619-9928-8ed9aed5ca16 admin admin] The token cache is being invalidate because project c190d29b390a4bf4a1153e756548ec45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:33:33.873073 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7cffaba4-447b-4619-9928-8ed9aed5ca16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:33.874579 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7cffaba4-447b-4619-9928-8ed9aed5ca16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:33.876888 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1320/2646] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:33:33 2022] DELETE /identity/v3/projects/c190d29b390a4bf4a1153e756548ec45 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:33.882000 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a076c36-2884-41f6-a915-7369c83eae59 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:33.882328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a076c36-2884-41f6-a915-7369c83eae59 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:33.882769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a076c36-2884-41f6-a915-7369c83eae59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:33.945143 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-0a076c36-2884-41f6-a915-7369c83eae59 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:33.946970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-0a076c36-2884-41f6-a915-7369c83eae59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:34.059572 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1327/2647] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:33:34.068995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923ac75-e038-4222-93ee-c6cadcddbd95 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:34.069403 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923ac75-e038-4222-93ee-c6cadcddbd95 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:34.069575 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923ac75-e038-4222-93ee-c6cadcddbd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:34.109159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-5923ac75-e038-4222-93ee-c6cadcddbd95 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:34.111741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-5923ac75-e038-4222-93ee-c6cadcddbd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:34.136310 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1321/2648] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:33:34.141188 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a076c36-2884-41f6-a915-7369c83eae59 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:34.161356 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a076c36-2884-41f6-a915-7369c83eae59 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:34.166987 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0b32a-a2a3-4ad0-b9d5-90671cdbb6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:34.168544 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0b32a-a2a3-4ad0-b9d5-90671cdbb6a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:34.170594 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0b32a-a2a3-4ad0-b9d5-90671cdbb6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:34.170594 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0b32a-a2a3-4ad0-b9d5-90671cdbb6a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:34.171435 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0b32a-a2a3-4ad0-b9d5-90671cdbb6a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:33:34.171435 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0b32a-a2a3-4ad0-b9d5-90671cdbb6a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:34.173500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef0b32a-a2a3-4ad0-b9d5-90671cdbb6a7 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:34.197778 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1328/2649] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:33:34.202272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923ac75-e038-4222-93ee-c6cadcddbd95 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:34.215677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923ac75-e038-4222-93ee-c6cadcddbd95 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:34.232907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1af3ca-3bce-4743-8fd1-c06bbe999c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:34.233637 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1af3ca-3bce-4743-8fd1-c06bbe999c7e admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:34.234326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1af3ca-3bce-4743-8fd1-c06bbe999c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:34.234620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1af3ca-3bce-4743-8fd1-c06bbe999c7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:34.235328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1af3ca-3bce-4743-8fd1-c06bbe999c7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:34.238243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1af3ca-3bce-4743-8fd1-c06bbe999c7e admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:34.323010 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1322/2650] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:33:34 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:33:34.327928 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0b32a-a2a3-4ad0-b9d5-90671cdbb6a7 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:34.352674 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1af3ca-3bce-4743-8fd1-c06bbe999c7e admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:34.360645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-cb1af3ca-3bce-4743-8fd1-c06bbe999c7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:34.362587 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0b32a-a2a3-4ad0-b9d5-90671cdbb6a7 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:34.385983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:34.529426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1af3ca-3bce-4743-8fd1-c06bbe999c7e admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:34.536705 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea7b60-b6bb-4044-a5bf-39aa56d9f0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:34.537523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea7b60-b6bb-4044-a5bf-39aa56d9f0d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:34.537921 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea7b60-b6bb-4044-a5bf-39aa56d9f0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:34.538355 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea7b60-b6bb-4044-a5bf-39aa56d9f0d5 admin admin] PATH_INFO: `/v3/users/de73f338e6ef4844979ead86ef2bd16c` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:34.541219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea7b60-b6bb-4044-a5bf-39aa56d9f0d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de73f338e6ef4844979ead86ef2bd16c)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:34.543556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea7b60-b6bb-4044-a5bf-39aa56d9f0d5 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:34.564205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:34.617217 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64981717-419c-4e45-84f4-5fecddf81a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:34.618578 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64981717-419c-4e45-84f4-5fecddf81a3d admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:34.620024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64981717-419c-4e45-84f4-5fecddf81a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:34.630048 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64981717-419c-4e45-84f4-5fecddf81a3d admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:34.633278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64981717-419c-4e45-84f4-5fecddf81a3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:34.635513 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64981717-419c-4e45-84f4-5fecddf81a3d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:34.647939 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-faea7b60-b6bb-4044-a5bf-39aa56d9f0d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de73f338e6ef4844979ead86ef2bd16c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:34.680267 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-faea7b60-b6bb-4044-a5bf-39aa56d9f0d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de73f338e6ef4844979ead86ef2bd16c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:34.692722 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1329/2651] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:33:34 2022] POST /identity/v3/users => generated 294 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:33:34.697772 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64981717-419c-4e45-84f4-5fecddf81a3d admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:34.712527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-faea7b60-b6bb-4044-a5bf-39aa56d9f0d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de73f338e6ef4844979ead86ef2bd16c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:34.713691 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-faea7b60-b6bb-4044-a5bf-39aa56d9f0d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de73f338e6ef4844979ead86ef2bd16c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:34.722169 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64981717-419c-4e45-84f4-5fecddf81a3d admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:34.730940 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-faea7b60-b6bb-4044-a5bf-39aa56d9f0d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de73f338e6ef4844979ead86ef2bd16c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:34.738782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b311a-0a35-4477-962c-ed3409196473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:34.739268 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b311a-0a35-4477-962c-ed3409196473 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:34.739759 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b311a-0a35-4477-962c-ed3409196473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:34.740431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b311a-0a35-4477-962c-ed3409196473 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:34.741564 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1323/2652] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:33:34 2022] DELETE /identity/v3/users/de73f338e6ef4844979ead86ef2bd16c => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:33:34.741910 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b311a-0a35-4477-962c-ed3409196473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:34.744892 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b311a-0a35-4477-962c-ed3409196473 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:34.762971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1330/2653] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:34.767741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea7b60-b6bb-4044-a5bf-39aa56d9f0d5 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:34.769595 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-faea7b60-b6bb-4044-a5bf-39aa56d9f0d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:34.880680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea7b60-b6bb-4044-a5bf-39aa56d9f0d5 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:34.885925 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88712fe6-fa10-4a30-ab4a-a4d4b242532a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:34.886762 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88712fe6-fa10-4a30-ab4a-a4d4b242532a admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:34.887796 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88712fe6-fa10-4a30-ab4a-a4d4b242532a admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:34.888241 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88712fe6-fa10-4a30-ab4a-a4d4b242532a admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:34.889017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88712fe6-fa10-4a30-ab4a-a4d4b242532a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:34.890901 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88712fe6-fa10-4a30-ab4a-a4d4b242532a admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:34.906530 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1324/2654] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:34.911628 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b311a-0a35-4477-962c-ed3409196473 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:34.926640 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b311a-0a35-4477-962c-ed3409196473 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:34.932894 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd757e-cb3c-427a-b646-f85ee3f6248c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:34.933780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd757e-cb3c-427a-b646-f85ee3f6248c admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:34.934158 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd757e-cb3c-427a-b646-f85ee3f6248c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:34.934534 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd757e-cb3c-427a-b646-f85ee3f6248c admin admin] PATH_INFO: `/v3/projects/4f167d60a99a413ca9beaea96a5e2d30/users/f99af449db304cd696116c51ee0df921/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:34.955929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd757e-cb3c-427a-b646-f85ee3f6248c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f167d60a99a413ca9beaea96a5e2d30, user_id=f99af449db304cd696116c51ee0df921, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:34.959908 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd757e-cb3c-427a-b646-f85ee3f6248c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:34.984893 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1331/2655] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:34 2022] PUT /identity/v3/projects/4f167d60a99a413ca9beaea96a5e2d30/users/f99af449db304cd696116c51ee0df921/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:34.999964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d22d6-7eb7-4405-acd9-a9b5c753b741 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:34.999964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d22d6-7eb7-4405-acd9-a9b5c753b741 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:34.999964 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d22d6-7eb7-4405-acd9-a9b5c753b741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:35.038125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-760d22d6-7eb7-4405-acd9-a9b5c753b741 None None] MFA Rules not processed for user `f99af449db304cd696116c51ee0df921`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:35.039570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-760d22d6-7eb7-4405-acd9-a9b5c753b741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:35.052049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:35.163756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1325/2656] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:35.168918 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd757e-cb3c-427a-b646-f85ee3f6248c admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:35.277798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd757e-cb3c-427a-b646-f85ee3f6248c admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:35.286286 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0c3bcd-9eed-4b08-a012-0b48bee1a632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:35.287671 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c3bcd-9eed-4b08-a012-0b48bee1a632 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:35.288307 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c3bcd-9eed-4b08-a012-0b48bee1a632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:35.288949 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c3bcd-9eed-4b08-a012-0b48bee1a632 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:35.289807 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c3bcd-9eed-4b08-a012-0b48bee1a632 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:35.292820 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0c3bcd-9eed-4b08-a012-0b48bee1a632 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:35.356079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1332/2657] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:33:35 2022] POST /identity/v3/projects => generated 387 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:35.362473 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d22d6-7eb7-4405-acd9-a9b5c753b741 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:35.394275 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d22d6-7eb7-4405-acd9-a9b5c753b741 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:35.483139 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dce79a-20e4-4b34-a104-806df8da1861 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:35.484550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dce79a-20e4-4b34-a104-806df8da1861 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:35.485740 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dce79a-20e4-4b34-a104-806df8da1861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:35.546982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-45dce79a-20e4-4b34-a104-806df8da1861 None None] MFA Rules not processed for user `61ff29258a214817b8fc15b51fe8b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:35.553551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-45dce79a-20e4-4b34-a104-806df8da1861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:35.568022 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15087f93-69be-4ac0-a30d-747616b75db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:35.569726 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15087f93-69be-4ac0-a30d-747616b75db5 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:35.570735 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15087f93-69be-4ac0-a30d-747616b75db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:35.573378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15087f93-69be-4ac0-a30d-747616b75db5 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:35.574535 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15087f93-69be-4ac0-a30d-747616b75db5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:35.580761 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15087f93-69be-4ac0-a30d-747616b75db5 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:35.641215 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1326/2658] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:33:35 2022] POST /identity/v3/users => generated 300 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:35.646131 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15087f93-69be-4ac0-a30d-747616b75db5 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:35.651169 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-15087f93-69be-4ac0-a30d-747616b75db5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:35.698597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15087f93-69be-4ac0-a30d-747616b75db5 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:35.711983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:35.713859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:35.714077 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1333/2659] 10.222.0.17 () {58 vars in 967 bytes} [Sun Aug 7 19:33:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:35.715117 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dce79a-20e4-4b34-a104-806df8da1861 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:35.716216 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:35.716946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe admin admin] PATH_INFO: `/v3/projects/34950b5c9a734f5cb5820e83dedd4030` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:35.722847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=34950b5c9a734f5cb5820e83dedd4030)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:35.731488 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dce79a-20e4-4b34-a104-806df8da1861 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:35.736566 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:35.737994 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b535971-c79b-4f42-95d1-ef554f0d8579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:35.738913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b535971-c79b-4f42-95d1-ef554f0d8579 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:35.739478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b535971-c79b-4f42-95d1-ef554f0d8579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:35.739983 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b535971-c79b-4f42-95d1-ef554f0d8579 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:35.740971 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b535971-c79b-4f42-95d1-ef554f0d8579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:35.744067 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b535971-c79b-4f42-95d1-ef554f0d8579 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:35.771820 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1334/2660] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:35.773560 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b535971-c79b-4f42-95d1-ef554f0d8579 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:35.777127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-3b535971-c79b-4f42-95d1-ef554f0d8579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:35.816694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:35.852645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34950b5c9a734f5cb5820e83dedd4030', 'request_id': 'req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:35.871047 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34950b5c9a734f5cb5820e83dedd4030', 'request_id': 'req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:35.894751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34950b5c9a734f5cb5820e83dedd4030', 'request_id': 'req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:35.926265 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe admin admin] The token cache is being invalidate because project 34950b5c9a734f5cb5820e83dedd4030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:33:35.926811 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:35.927462 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:35.940035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1327/2661] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:33:35 2022] DELETE /identity/v3/projects/34950b5c9a734f5cb5820e83dedd4030 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:35.940035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:35.940035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:36.049416 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b535971-c79b-4f42-95d1-ef554f0d8579 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:36.081759 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:36.234648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61023c5-9222-47f5-9232-d5d3cc742416 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db51971158a946c98a8ed716ecc5d72b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:36.238071 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61023c5-9222-47f5-9232-d5d3cc742416 service swift] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:36.238537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61023c5-9222-47f5-9232-d5d3cc742416 service swift] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:36.239338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61023c5-9222-47f5-9232-d5d3cc742416 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:36.243385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c62bf4d-7891-49f7-8ab0-e3ece324c4fe admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:36.244824 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d61023c5-9222-47f5-9232-d5d3cc742416 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:36.255487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa62968-eacb-48a2-b5d7-ff9737c884b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:36.257908 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa62968-eacb-48a2-b5d7-ff9737c884b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:36.258723 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa62968-eacb-48a2-b5d7-ff9737c884b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:36.259526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa62968-eacb-48a2-b5d7-ff9737c884b0 admin admin] PATH_INFO: `/v3/projects/f4745d417db5477c8fdd370943eb4348/users/5a58a81be37440be924e3b5d685b7b65/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:36.312535 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa62968-eacb-48a2-b5d7-ff9737c884b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4745d417db5477c8fdd370943eb4348, user_id=5a58a81be37440be924e3b5d685b7b65, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:36.328420 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa62968-eacb-48a2-b5d7-ff9737c884b0 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:36.360038 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1328/2662] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:35 2022] PUT /identity/v3/projects/f4745d417db5477c8fdd370943eb4348/users/5a58a81be37440be924e3b5d685b7b65/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:36.365946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa62968-eacb-48a2-b5d7-ff9737c884b0 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:36.401945 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61023c5-9222-47f5-9232-d5d3cc742416 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:36.406755 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61023c5-9222-47f5-9232-d5d3cc742416 service swift] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:36.454772 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1335/2663] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Aug 7 19:33:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 19:33:36.508262 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa62968-eacb-48a2-b5d7-ff9737c884b0 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:36.527635 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab6ff0-b485-4aa1-b193-da5342a54557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:36.528447 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab6ff0-b485-4aa1-b193-da5342a54557 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:36.528828 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab6ff0-b485-4aa1-b193-da5342a54557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:36.529315 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab6ff0-b485-4aa1-b193-da5342a54557 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:36.530173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab6ff0-b485-4aa1-b193-da5342a54557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:36.532281 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab6ff0-b485-4aa1-b193-da5342a54557 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:36.556770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1329/2664] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:36.558744 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61023c5-9222-47f5-9232-d5d3cc742416 service swift] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:36.586672 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61023c5-9222-47f5-9232-d5d3cc742416 service swift] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:36.604497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec92fa-f4fa-43e6-88df-9a4f4587f61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:36.606141 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec92fa-f4fa-43e6-88df-9a4f4587f61f admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:36.609017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec92fa-f4fa-43e6-88df-9a4f4587f61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:36.609316 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec92fa-f4fa-43e6-88df-9a4f4587f61f admin admin] PATH_INFO: `/v3/projects/f4745d417db5477c8fdd370943eb4348/users/5a58a81be37440be924e3b5d685b7b65/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:36.617716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec92fa-f4fa-43e6-88df-9a4f4587f61f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4745d417db5477c8fdd370943eb4348, user_id=5a58a81be37440be924e3b5d685b7b65, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:36.625861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec92fa-f4fa-43e6-88df-9a4f4587f61f admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:36.680765 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1336/2665] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:36 2022] PUT /identity/v3/projects/f4745d417db5477c8fdd370943eb4348/users/5a58a81be37440be924e3b5d685b7b65/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:36.699149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e349b-bcb4-47d5-8460-10c33e9fce98 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:36.699645 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e349b-bcb4-47d5-8460-10c33e9fce98 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:36.700177 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e349b-bcb4-47d5-8460-10c33e9fce98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:36.768875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-c35e349b-bcb4-47d5-8460-10c33e9fce98 None None] MFA Rules not processed for user `5a58a81be37440be924e3b5d685b7b65`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:36.776367 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c35e349b-bcb4-47d5-8460-10c33e9fce98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:37.027911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1330/2666] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:37.037080 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec92fa-f4fa-43e6-88df-9a4f4587f61f admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:37.039807 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d5ec92fa-f4fa-43e6-88df-9a4f4587f61f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:37.121775 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:37.129222 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec92fa-f4fa-43e6-88df-9a4f4587f61f admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:37.136359 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef8b82-4cc4-4761-889c-39ec528d3043 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:37.137894 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef8b82-4cc4-4761-889c-39ec528d3043 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:37.139172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef8b82-4cc4-4761-889c-39ec528d3043 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:37.139704 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef8b82-4cc4-4761-889c-39ec528d3043 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:37.151636 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef8b82-4cc4-4761-889c-39ec528d3043 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:37.153695 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef8b82-4cc4-4761-889c-39ec528d3043 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:37.166216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1337/2667] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:33:37.202899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e349b-bcb4-47d5-8460-10c33e9fce98 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:37.290854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:37.358079 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e349b-bcb4-47d5-8460-10c33e9fce98 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:37.368834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bc723-f9c9-4422-a5aa-684d7ffc6673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:37.368834 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc723-f9c9-4422-a5aa-684d7ffc6673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:37.369969 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc723-f9c9-4422-a5aa-684d7ffc6673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:37.369969 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3bc723-f9c9-4422-a5aa-684d7ffc6673 admin admin] PATH_INFO: `/v3/users/f99af449db304cd696116c51ee0df921` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:37.372034 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bc723-f9c9-4422-a5aa-684d7ffc6673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f99af449db304cd696116c51ee0df921)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:37.376613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3bc723-f9c9-4422-a5aa-684d7ffc6673 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:37.504446 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cc3bc723-f9c9-4422-a5aa-684d7ffc6673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f99af449db304cd696116c51ee0df921'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:37.521636 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cc3bc723-f9c9-4422-a5aa-684d7ffc6673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f99af449db304cd696116c51ee0df921'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:37.564545 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cc3bc723-f9c9-4422-a5aa-684d7ffc6673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f99af449db304cd696116c51ee0df921'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:37.566323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cc3bc723-f9c9-4422-a5aa-684d7ffc6673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f99af449db304cd696116c51ee0df921'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:37.586991 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-cc3bc723-f9c9-4422-a5aa-684d7ffc6673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f99af449db304cd696116c51ee0df921'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:37.601761 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1331/2668] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:33:37 2022] DELETE /identity/v3/users/f99af449db304cd696116c51ee0df921 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:37.610537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b6435-2bcf-4811-9f9b-aa7fabcc9f59 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:37.611108 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b6435-2bcf-4811-9f9b-aa7fabcc9f59 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:37.613944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b6435-2bcf-4811-9f9b-aa7fabcc9f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:37.664051 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-253b6435-2bcf-4811-9f9b-aa7fabcc9f59 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:37.665519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-253b6435-2bcf-4811-9f9b-aa7fabcc9f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:37.804418 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1338/2669] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:37.848571 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bc723-f9c9-4422-a5aa-684d7ffc6673 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:37.851847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-cc3bc723-f9c9-4422-a5aa-684d7ffc6673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:38.147193 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3bc723-f9c9-4422-a5aa-684d7ffc6673 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:38.162779 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2206d-20c0-47b7-bf81-2e7f2dac21ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:38.163590 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2206d-20c0-47b7-bf81-2e7f2dac21ba service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:38.163880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2206d-20c0-47b7-bf81-2e7f2dac21ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:38.164181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c2206d-20c0-47b7-bf81-2e7f2dac21ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:38.189589 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c2206d-20c0-47b7-bf81-2e7f2dac21ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:38.191181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c2206d-20c0-47b7-bf81-2e7f2dac21ba service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:38.208016 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1332/2670] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:33:39.100487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b6435-2bcf-4811-9f9b-aa7fabcc9f59 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:39.104716 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-253b6435-2bcf-4811-9f9b-aa7fabcc9f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:39.169467 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b6435-2bcf-4811-9f9b-aa7fabcc9f59 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:39.183473 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bbae3-8350-4cd6-b986-41d92aabebd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:39.184773 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bbae3-8350-4cd6-b986-41d92aabebd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:39.185321 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bbae3-8350-4cd6-b986-41d92aabebd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:39.185861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bbae3-8350-4cd6-b986-41d92aabebd0 admin admin] PATH_INFO: `/v3/projects/4f167d60a99a413ca9beaea96a5e2d30` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:39.187673 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6bbae3-8350-4cd6-b986-41d92aabebd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f167d60a99a413ca9beaea96a5e2d30)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:39.190260 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6bbae3-8350-4cd6-b986-41d92aabebd0 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:39.271103 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0a6bbae3-8350-4cd6-b986-41d92aabebd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f167d60a99a413ca9beaea96a5e2d30', 'request_id': 'req-0a6bbae3-8350-4cd6-b986-41d92aabebd0'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:39.279763 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0a6bbae3-8350-4cd6-b986-41d92aabebd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f167d60a99a413ca9beaea96a5e2d30', 'request_id': 'req-0a6bbae3-8350-4cd6-b986-41d92aabebd0'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:39.291053 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0a6bbae3-8350-4cd6-b986-41d92aabebd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f167d60a99a413ca9beaea96a5e2d30', 'request_id': 'req-0a6bbae3-8350-4cd6-b986-41d92aabebd0'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:39.307483 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0a6bbae3-8350-4cd6-b986-41d92aabebd0 admin admin] The token cache is being invalidate because project 4f167d60a99a413ca9beaea96a5e2d30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:33:39.308148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0a6bbae3-8350-4cd6-b986-41d92aabebd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:39.309125 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-0a6bbae3-8350-4cd6-b986-41d92aabebd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:39.311055 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1339/2671] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:33:39 2022] DELETE /identity/v3/projects/4f167d60a99a413ca9beaea96a5e2d30 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:39.315345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2206d-20c0-47b7-bf81-2e7f2dac21ba service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:39.327049 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c2206d-20c0-47b7-bf81-2e7f2dac21ba service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:39.386026 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:39.400696 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81531680-20e3-4a22-8d88-f14c49e5672c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:39.401554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81531680-20e3-4a22-8d88-f14c49e5672c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:39.401919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81531680-20e3-4a22-8d88-f14c49e5672c admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:39.402294 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81531680-20e3-4a22-8d88-f14c49e5672c admin admin] PATH_INFO: `/v3/users/5a58a81be37440be924e3b5d685b7b65` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:39.403765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81531680-20e3-4a22-8d88-f14c49e5672c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a58a81be37440be924e3b5d685b7b65)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:39.408179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81531680-20e3-4a22-8d88-f14c49e5672c admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:39.481256 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-81531680-20e3-4a22-8d88-f14c49e5672c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a58a81be37440be924e3b5d685b7b65'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:39.495076 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-81531680-20e3-4a22-8d88-f14c49e5672c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a58a81be37440be924e3b5d685b7b65'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:39.526016 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-81531680-20e3-4a22-8d88-f14c49e5672c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a58a81be37440be924e3b5d685b7b65'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:39.527222 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-81531680-20e3-4a22-8d88-f14c49e5672c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a58a81be37440be924e3b5d685b7b65'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:39.541730 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-81531680-20e3-4a22-8d88-f14c49e5672c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a58a81be37440be924e3b5d685b7b65'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:39.550101 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1333/2672] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:33:39 2022] DELETE /identity/v3/users/5a58a81be37440be924e3b5d685b7b65 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:40.125920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bbae3-8350-4cd6-b986-41d92aabebd0 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:40.129110 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-0a6bbae3-8350-4cd6-b986-41d92aabebd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:40.195306 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bbae3-8350-4cd6-b986-41d92aabebd0 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:40.370359 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47021d6-65d9-479d-9f2a-a81c61f9d3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:40.371039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47021d6-65d9-479d-9f2a-a81c61f9d3ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:40.371583 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47021d6-65d9-479d-9f2a-a81c61f9d3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:40.371862 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47021d6-65d9-479d-9f2a-a81c61f9d3ad admin admin] PATH_INFO: `/v3/projects/f4745d417db5477c8fdd370943eb4348` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:40.374152 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47021d6-65d9-479d-9f2a-a81c61f9d3ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4745d417db5477c8fdd370943eb4348)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:40.376617 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47021d6-65d9-479d-9f2a-a81c61f9d3ad admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:40.407405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:40.492399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e47021d6-65d9-479d-9f2a-a81c61f9d3ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4745d417db5477c8fdd370943eb4348', 'request_id': 'req-e47021d6-65d9-479d-9f2a-a81c61f9d3ad'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:40.512258 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e47021d6-65d9-479d-9f2a-a81c61f9d3ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4745d417db5477c8fdd370943eb4348', 'request_id': 'req-e47021d6-65d9-479d-9f2a-a81c61f9d3ad'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:40.537409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e47021d6-65d9-479d-9f2a-a81c61f9d3ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4745d417db5477c8fdd370943eb4348', 'request_id': 'req-e47021d6-65d9-479d-9f2a-a81c61f9d3ad'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:40.572337 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e47021d6-65d9-479d-9f2a-a81c61f9d3ad admin admin] The token cache is being invalidate because project f4745d417db5477c8fdd370943eb4348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:33:40.572653 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e47021d6-65d9-479d-9f2a-a81c61f9d3ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:40.573661 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-e47021d6-65d9-479d-9f2a-a81c61f9d3ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:40.575573 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1340/2673] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:33:40 2022] DELETE /identity/v3/projects/f4745d417db5477c8fdd370943eb4348 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:40.581015 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d85864-e01e-4dca-8aea-f73546d30886 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:40.581678 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d85864-e01e-4dca-8aea-f73546d30886 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:40.581678 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d85864-e01e-4dca-8aea-f73546d30886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:40.604738 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-d5d85864-e01e-4dca-8aea-f73546d30886 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:40.606378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-d5d85864-e01e-4dca-8aea-f73546d30886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:40.707512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1bbc8-c0ba-4c22-b443-648f024072e9 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:40.707734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1bbc8-c0ba-4c22-b443-648f024072e9 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:40.707967 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1bbc8-c0ba-4c22-b443-648f024072e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:40.709967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1334/2674] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:40.755887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-baa1bbc8-c0ba-4c22-b443-648f024072e9 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:40.760602 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-baa1bbc8-c0ba-4c22-b443-648f024072e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:40.794409 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1341/2675] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:40.797809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d85864-e01e-4dca-8aea-f73546d30886 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:40.817385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d85864-e01e-4dca-8aea-f73546d30886 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:40.838991 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc8dd4-3073-4770-bbe2-07976aac2e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:40.840247 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc8dd4-3073-4770-bbe2-07976aac2e72 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:40.840845 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc8dd4-3073-4770-bbe2-07976aac2e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:40.841885 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc8dd4-3073-4770-bbe2-07976aac2e72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:40.843193 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc8dd4-3073-4770-bbe2-07976aac2e72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:33:40.844394 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc8dd4-3073-4770-bbe2-07976aac2e72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:40.846966 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc8dd4-3073-4770-bbe2-07976aac2e72 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:40.888756 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1335/2676] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:33:40.901946 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1bbc8-c0ba-4c22-b443-648f024072e9 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:40.930272 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1bbc8-c0ba-4c22-b443-648f024072e9 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:40.943270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e4bf0-846a-481c-917a-19debc7462d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:40.946920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e4bf0-846a-481c-917a-19debc7462d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:40.947419 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e4bf0-846a-481c-917a-19debc7462d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:40.947892 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e4bf0-846a-481c-917a-19debc7462d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:40.948841 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e4bf0-846a-481c-917a-19debc7462d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:40.952075 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e4bf0-846a-481c-917a-19debc7462d9 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:41.014087 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc8dd4-3073-4770-bbe2-07976aac2e72 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:41.014463 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1342/2677] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:33:40 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:41.030057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc8dd4-3073-4770-bbe2-07976aac2e72 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:41.153342 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:41.165481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9622e-feb3-4551-a682-09cc284fa194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:41.166699 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9622e-feb3-4551-a682-09cc284fa194 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:41.167421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9622e-feb3-4551-a682-09cc284fa194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:41.167932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9622e-feb3-4551-a682-09cc284fa194 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:41.169219 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9622e-feb3-4551-a682-09cc284fa194 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:41.174272 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9622e-feb3-4551-a682-09cc284fa194 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:41.213345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1336/2678] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:33:41 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:41.217329 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e4bf0-846a-481c-917a-19debc7462d9 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:41.242248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e4bf0-846a-481c-917a-19debc7462d9 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:41.247985 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485291c-0e2c-489e-8d0a-85ed3b0d7177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:41.248727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485291c-0e2c-489e-8d0a-85ed3b0d7177 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:41.249258 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485291c-0e2c-489e-8d0a-85ed3b0d7177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:41.249584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485291c-0e2c-489e-8d0a-85ed3b0d7177 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:41.250270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485291c-0e2c-489e-8d0a-85ed3b0d7177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:41.252288 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485291c-0e2c-489e-8d0a-85ed3b0d7177 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:41.269707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1343/2679] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:41.275425 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9622e-feb3-4551-a682-09cc284fa194 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:41.289894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9622e-feb3-4551-a682-09cc284fa194 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:41.297016 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11194b-4e71-4a0f-9b95-84970c61aa53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:41.297977 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11194b-4e71-4a0f-9b95-84970c61aa53 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:41.298390 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11194b-4e71-4a0f-9b95-84970c61aa53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:41.298799 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11194b-4e71-4a0f-9b95-84970c61aa53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:41.299871 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11194b-4e71-4a0f-9b95-84970c61aa53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:41.302324 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11194b-4e71-4a0f-9b95-84970c61aa53 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:41.315061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1337/2680] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:41.319795 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485291c-0e2c-489e-8d0a-85ed3b0d7177 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:41.332285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485291c-0e2c-489e-8d0a-85ed3b0d7177 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:41.339130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44548cc0-8a7e-4748-89ce-832327237e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:41.340083 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44548cc0-8a7e-4748-89ce-832327237e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:41.340972 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44548cc0-8a7e-4748-89ce-832327237e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:41.341281 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44548cc0-8a7e-4748-89ce-832327237e97 admin admin] PATH_INFO: `/v3/projects/63336c2d268147e7b8c2bb928273a41d/users/b2f60092b2bb4d499760e39686f402e0/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:41.360581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44548cc0-8a7e-4748-89ce-832327237e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63336c2d268147e7b8c2bb928273a41d, user_id=b2f60092b2bb4d499760e39686f402e0, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:41.363573 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44548cc0-8a7e-4748-89ce-832327237e97 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:41.380057 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1344/2681] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:41 2022] PUT /identity/v3/projects/63336c2d268147e7b8c2bb928273a41d/users/b2f60092b2bb4d499760e39686f402e0/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:33:41.391890 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522621fd-779b-4f31-bec5-75712e2e3129 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:41.392460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522621fd-779b-4f31-bec5-75712e2e3129 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:41.392748 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522621fd-779b-4f31-bec5-75712e2e3129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:41.430001 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-522621fd-779b-4f31-bec5-75712e2e3129 None None] MFA Rules not processed for user `b2f60092b2bb4d499760e39686f402e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:41.431428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-522621fd-779b-4f31-bec5-75712e2e3129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:41.593904 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1338/2682] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:41.594916 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44548cc0-8a7e-4748-89ce-832327237e97 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:41.711493 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44548cc0-8a7e-4748-89ce-832327237e97 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:41.720679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e606b2-c55b-4290-bb7f-4abc903d71a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:41.721853 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e606b2-c55b-4290-bb7f-4abc903d71a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:41.722530 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e606b2-c55b-4290-bb7f-4abc903d71a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:41.723129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e606b2-c55b-4290-bb7f-4abc903d71a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:41.724450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e606b2-c55b-4290-bb7f-4abc903d71a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:41.729148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e606b2-c55b-4290-bb7f-4abc903d71a6 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:41.798729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1345/2683] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:33:41 2022] POST /identity/v3/projects => generated 377 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:41.803540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522621fd-779b-4f31-bec5-75712e2e3129 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:41.850723 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522621fd-779b-4f31-bec5-75712e2e3129 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:42.069414 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251a1d2-95a7-4997-9ff8-e7936348d459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:42.070718 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251a1d2-95a7-4997-9ff8-e7936348d459 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:42.071339 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251a1d2-95a7-4997-9ff8-e7936348d459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:42.073160 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251a1d2-95a7-4997-9ff8-e7936348d459 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:42.074641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251a1d2-95a7-4997-9ff8-e7936348d459 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:42.078739 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251a1d2-95a7-4997-9ff8-e7936348d459 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:42.136730 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e606b2-c55b-4290-bb7f-4abc903d71a6 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:42.139823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1339/2684] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:33:41 2022] POST /identity/v3/users => generated 295 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:42.160219 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e606b2-c55b-4290-bb7f-4abc903d71a6 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:42.173163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992de7d7-b045-4271-9d40-a1794198b5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:42.176181 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992de7d7-b045-4271-9d40-a1794198b5aa admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:42.177052 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992de7d7-b045-4271-9d40-a1794198b5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:42.177326 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992de7d7-b045-4271-9d40-a1794198b5aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:42.178518 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992de7d7-b045-4271-9d40-a1794198b5aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:42.181851 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992de7d7-b045-4271-9d40-a1794198b5aa admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:42.204591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1346/2685] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:42.210815 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251a1d2-95a7-4997-9ff8-e7936348d459 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:42.231811 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251a1d2-95a7-4997-9ff8-e7936348d459 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:42.252477 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc156a-d767-400b-952c-802a019894fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:42.255820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc156a-d767-400b-952c-802a019894fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:42.256586 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc156a-d767-400b-952c-802a019894fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:42.257148 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc156a-d767-400b-952c-802a019894fc admin admin] PATH_INFO: `/v3/projects/62cfd3ed2c51448bb1ff2873cfde7479/users/a1da1de903b64a96bcebed6aeb114bbb/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:42.297910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc156a-d767-400b-952c-802a019894fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=62cfd3ed2c51448bb1ff2873cfde7479, user_id=a1da1de903b64a96bcebed6aeb114bbb, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:42.303577 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc156a-d767-400b-952c-802a019894fc admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:42.341641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1340/2686] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:42 2022] PUT /identity/v3/projects/62cfd3ed2c51448bb1ff2873cfde7479/users/a1da1de903b64a96bcebed6aeb114bbb/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:42.342396 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992de7d7-b045-4271-9d40-a1794198b5aa admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:42.368205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992de7d7-b045-4271-9d40-a1794198b5aa admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:42.392979 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:42.514627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64a011-8674-4ede-bfff-c5ae3f5e36f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:42.516064 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64a011-8674-4ede-bfff-c5ae3f5e36f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:42.516836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64a011-8674-4ede-bfff-c5ae3f5e36f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:42.517494 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64a011-8674-4ede-bfff-c5ae3f5e36f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:42.519957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64a011-8674-4ede-bfff-c5ae3f5e36f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:42.523037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64a011-8674-4ede-bfff-c5ae3f5e36f3 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:42.567897 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1347/2687] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:42.574325 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc156a-d767-400b-952c-802a019894fc admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:42.591345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc156a-d767-400b-952c-802a019894fc admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:42.598718 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045c740-817c-44a8-98f4-d19924c70fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:42.600926 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045c740-817c-44a8-98f4-d19924c70fc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:42.600926 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045c740-817c-44a8-98f4-d19924c70fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:42.600926 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045c740-817c-44a8-98f4-d19924c70fc5 admin admin] PATH_INFO: `/v3/projects/62cfd3ed2c51448bb1ff2873cfde7479/users/a1da1de903b64a96bcebed6aeb114bbb/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:42.604812 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045c740-817c-44a8-98f4-d19924c70fc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62cfd3ed2c51448bb1ff2873cfde7479, user_id=a1da1de903b64a96bcebed6aeb114bbb, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:42.608981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045c740-817c-44a8-98f4-d19924c70fc5 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:42.627828 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1341/2688] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:42 2022] PUT /identity/v3/projects/62cfd3ed2c51448bb1ff2873cfde7479/users/a1da1de903b64a96bcebed6aeb114bbb/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:33:42.648371 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d0c5a-0265-44d6-a0e9-58e63f390be7 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:42.648935 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d0c5a-0265-44d6-a0e9-58e63f390be7 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:42.650205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d0c5a-0265-44d6-a0e9-58e63f390be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:42.710371 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-ca3d0c5a-0265-44d6-a0e9-58e63f390be7 None None] MFA Rules not processed for user `a1da1de903b64a96bcebed6aeb114bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:42.712549 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ca3d0c5a-0265-44d6-a0e9-58e63f390be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:42.875255 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1348/2689] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:42.892341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045c740-817c-44a8-98f4-d19924c70fc5 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:42.895713 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9045c740-817c-44a8-98f4-d19924c70fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:42.945291 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:43.057542 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045c740-817c-44a8-98f4-d19924c70fc5 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:43.074979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbc22a-995e-42e4-96e5-6984e28c7970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:43.074979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbc22a-995e-42e4-96e5-6984e28c7970 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:43.074979 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbc22a-995e-42e4-96e5-6984e28c7970 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:43.076109 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbc22a-995e-42e4-96e5-6984e28c7970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:43.094745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbc22a-995e-42e4-96e5-6984e28c7970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:43.096563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbc22a-995e-42e4-96e5-6984e28c7970 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:43.109460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1342/2690] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:33:43.895161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d0c5a-0265-44d6-a0e9-58e63f390be7 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:43.944098 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d0c5a-0265-44d6-a0e9-58e63f390be7 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:43.961338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bab565-1805-47f9-9c72-a561ddc967e6 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1da1de903b64a96bcebed6aeb114bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62cfd3ed2c51448bb1ff2873cfde7479', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:43.962370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bab565-1805-47f9-9c72-a561ddc967e6 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:43.966492 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bab565-1805-47f9-9c72-a561ddc967e6 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:43.967086 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bab565-1805-47f9-9c72-a561ddc967e6 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:43.968529 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bab565-1805-47f9-9c72-a561ddc967e6 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:33:43.972641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bab565-1805-47f9-9c72-a561ddc967e6 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:43.974802 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bab565-1805-47f9-9c72-a561ddc967e6 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:44.025217 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1349/2691] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:33:44.029816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbc22a-995e-42e4-96e5-6984e28c7970 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:44.041695 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbc22a-995e-42e4-96e5-6984e28c7970 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:44.050910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58074e85-bd08-45bb-aeeb-f6ca442b68ee tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1da1de903b64a96bcebed6aeb114bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62cfd3ed2c51448bb1ff2873cfde7479', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:44.051826 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58074e85-bd08-45bb-aeeb-f6ca442b68ee tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:44.052144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58074e85-bd08-45bb-aeeb-f6ca442b68ee tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:44.052582 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58074e85-bd08-45bb-aeeb-f6ca442b68ee tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:44.053458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58074e85-bd08-45bb-aeeb-f6ca442b68ee tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:44.055691 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58074e85-bd08-45bb-aeeb-f6ca442b68ee tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:44.122885 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1343/2692] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:33:44 2022] POST /identity/v3/projects => generated 354 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:44.127821 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bab565-1805-47f9-9c72-a561ddc967e6 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:44.141823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bab565-1805-47f9-9c72-a561ddc967e6 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:44.316604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85dc361-04e5-4deb-a6bf-2927aeff785e tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1da1de903b64a96bcebed6aeb114bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62cfd3ed2c51448bb1ff2873cfde7479', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:44.317270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85dc361-04e5-4deb-a6bf-2927aeff785e tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:44.317597 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85dc361-04e5-4deb-a6bf-2927aeff785e tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:44.317867 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85dc361-04e5-4deb-a6bf-2927aeff785e tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:44.318848 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85dc361-04e5-4deb-a6bf-2927aeff785e tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:33:44.318848 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85dc361-04e5-4deb-a6bf-2927aeff785e tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:44.328171 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85dc361-04e5-4deb-a6bf-2927aeff785e tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:44.365359 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1350/2693] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:33:44.487302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58074e85-bd08-45bb-aeeb-f6ca442b68ee tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:44.502137 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58074e85-bd08-45bb-aeeb-f6ca442b68ee tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:44.511560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585226c-645c-4a6e-bb17-66efc44307c2 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1da1de903b64a96bcebed6aeb114bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '62cfd3ed2c51448bb1ff2873cfde7479', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:44.520416 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585226c-645c-4a6e-bb17-66efc44307c2 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:44.521245 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585226c-645c-4a6e-bb17-66efc44307c2 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:44.521245 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585226c-645c-4a6e-bb17-66efc44307c2 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] PATH_INFO: `/v3/projects/e3a9851781054a61b0fd661ec2fc4e01` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:44.525842 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585226c-645c-4a6e-bb17-66efc44307c2 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e3a9851781054a61b0fd661ec2fc4e01)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:44.529471 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585226c-645c-4a6e-bb17-66efc44307c2 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:44.648849 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-f585226c-645c-4a6e-bb17-66efc44307c2 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3a9851781054a61b0fd661ec2fc4e01', 'request_id': 'req-f585226c-645c-4a6e-bb17-66efc44307c2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:44.663390 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-f585226c-645c-4a6e-bb17-66efc44307c2 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3a9851781054a61b0fd661ec2fc4e01', 'request_id': 'req-f585226c-645c-4a6e-bb17-66efc44307c2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:44.680817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85dc361-04e5-4deb-a6bf-2927aeff785e tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:44.682688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d85dc361-04e5-4deb-a6bf-2927aeff785e tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:44.685455 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-f585226c-645c-4a6e-bb17-66efc44307c2 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3a9851781054a61b0fd661ec2fc4e01', 'request_id': 'req-f585226c-645c-4a6e-bb17-66efc44307c2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:44.708607 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-f585226c-645c-4a6e-bb17-66efc44307c2 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] The token cache is being invalidate because project e3a9851781054a61b0fd661ec2fc4e01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:33:44.709598 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-f585226c-645c-4a6e-bb17-66efc44307c2 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:44.710577 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-f585226c-645c-4a6e-bb17-66efc44307c2 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:44.713199 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1344/2694] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:33:44 2022] DELETE /identity/v3/projects/e3a9851781054a61b0fd661ec2fc4e01 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:33:44.733121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:44.797751 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85dc361-04e5-4deb-a6bf-2927aeff785e tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:44.904182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c63bfc-dc2c-4561-bf6f-7f2ed34a60a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:44.905857 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c63bfc-dc2c-4561-bf6f-7f2ed34a60a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:44.906310 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c63bfc-dc2c-4561-bf6f-7f2ed34a60a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:44.906745 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c63bfc-dc2c-4561-bf6f-7f2ed34a60a3 admin admin] PATH_INFO: `/v3/users/dbec912e4d5c4c6aabed5a98b6aa632d` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:44.908619 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c63bfc-dc2c-4561-bf6f-7f2ed34a60a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbec912e4d5c4c6aabed5a98b6aa632d)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:44.910989 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c63bfc-dc2c-4561-bf6f-7f2ed34a60a3 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:45.028179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-20c63bfc-dc2c-4561-bf6f-7f2ed34a60a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbec912e4d5c4c6aabed5a98b6aa632d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:45.049189 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-20c63bfc-dc2c-4561-bf6f-7f2ed34a60a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbec912e4d5c4c6aabed5a98b6aa632d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:45.064325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-20c63bfc-dc2c-4561-bf6f-7f2ed34a60a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbec912e4d5c4c6aabed5a98b6aa632d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:45.066251 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-20c63bfc-dc2c-4561-bf6f-7f2ed34a60a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbec912e4d5c4c6aabed5a98b6aa632d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:45.066982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-20c63bfc-dc2c-4561-bf6f-7f2ed34a60a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbec912e4d5c4c6aabed5a98b6aa632d'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:45.091253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1351/2695] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:33:44 2022] DELETE /identity/v3/users/dbec912e4d5c4c6aabed5a98b6aa632d => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:45.129142 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585226c-645c-4a6e-bb17-66efc44307c2 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:45.131108 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f585226c-645c-4a6e-bb17-66efc44307c2 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:45.149676 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585226c-645c-4a6e-bb17-66efc44307c2 tempest-VolumeQuotaClassesTest-67042934 tempest-VolumeQuotaClassesTest-67042934-project-admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:45.240344 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4c346-ae8c-464f-8713-e1b84afb2b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:45.241319 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4c346-ae8c-464f-8713-e1b84afb2b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:45.253039 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4c346-ae8c-464f-8713-e1b84afb2b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:45.253393 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4c346-ae8c-464f-8713-e1b84afb2b67 admin admin] PATH_INFO: `/v3/users/b2f60092b2bb4d499760e39686f402e0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:45.255395 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4c346-ae8c-464f-8713-e1b84afb2b67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2f60092b2bb4d499760e39686f402e0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:45.260095 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4c346-ae8c-464f-8713-e1b84afb2b67 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:45.310568 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:45.362802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-66d4c346-ae8c-464f-8713-e1b84afb2b67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2f60092b2bb4d499760e39686f402e0'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:45.371772 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154578f-8c55-4693-aaca-eed02f2a28b9 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:45.372259 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154578f-8c55-4693-aaca-eed02f2a28b9 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:45.372476 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154578f-8c55-4693-aaca-eed02f2a28b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:45.384824 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-66d4c346-ae8c-464f-8713-e1b84afb2b67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2f60092b2bb4d499760e39686f402e0'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:45.414919 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-f154578f-8c55-4693-aaca-eed02f2a28b9 None None] MFA Rules not processed for user `e58ea4f0b3414240ab7eb521e7b92db1`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:45.417042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f154578f-8c55-4693-aaca-eed02f2a28b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:45.418844 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-66d4c346-ae8c-464f-8713-e1b84afb2b67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2f60092b2bb4d499760e39686f402e0'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:45.420439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-66d4c346-ae8c-464f-8713-e1b84afb2b67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2f60092b2bb4d499760e39686f402e0'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:45.432563 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-66d4c346-ae8c-464f-8713-e1b84afb2b67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2f60092b2bb4d499760e39686f402e0'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:45.443683 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1345/2696] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:33:45 2022] DELETE /identity/v3/users/b2f60092b2bb4d499760e39686f402e0 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:45.445959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e176a-e838-41bd-a7b6-f6fabbc4eec6 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:45.446321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e176a-e838-41bd-a7b6-f6fabbc4eec6 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:45.446641 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e176a-e838-41bd-a7b6-f6fabbc4eec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:45.475820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-721e176a-e838-41bd-a7b6-f6fabbc4eec6 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:45.476266 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-721e176a-e838-41bd-a7b6-f6fabbc4eec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:45.491651 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:45.555281 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1352/2697] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:33:45.556343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154578f-8c55-4693-aaca-eed02f2a28b9 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:45.558256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f154578f-8c55-4693-aaca-eed02f2a28b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:45.571183 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1346/2698] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:33:45.573336 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e176a-e838-41bd-a7b6-f6fabbc4eec6 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:45.575712 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-721e176a-e838-41bd-a7b6-f6fabbc4eec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:45.586392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154578f-8c55-4693-aaca-eed02f2a28b9 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:45.594827 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:45.595859 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:45.596420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:45.596906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3 admin admin] PATH_INFO: `/v3/projects/6e5e8fba8d65419393ba9829cafdd5b9` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:45.598883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e5e8fba8d65419393ba9829cafdd5b9)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:45.602444 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:45.674581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e5e8fba8d65419393ba9829cafdd5b9', 'request_id': 'req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:45.683161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e5e8fba8d65419393ba9829cafdd5b9', 'request_id': 'req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:45.693921 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e176a-e838-41bd-a7b6-f6fabbc4eec6 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:45.694237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e5e8fba8d65419393ba9829cafdd5b9', 'request_id': 'req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:45.707208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3 admin admin] The token cache is being invalidate because project 6e5e8fba8d65419393ba9829cafdd5b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:33:45.707559 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:45.708207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:45.709792 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1353/2699] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:33:45 2022] DELETE /identity/v3/projects/6e5e8fba8d65419393ba9829cafdd5b9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:45.713789 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:45.715241 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:45.752464 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:45.785290 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29f69d-0e7e-4426-b3f1-0607d0b374e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:45.785973 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29f69d-0e7e-4426-b3f1-0607d0b374e7 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:45.786282 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29f69d-0e7e-4426-b3f1-0607d0b374e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:45.786564 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29f69d-0e7e-4426-b3f1-0607d0b374e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:45.788851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0d29f69d-0e7e-4426-b3f1-0607d0b374e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:45.797688 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62ae98-c6fb-4cd0-b8bc-50bc19f560a3 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:45.802556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a942dfa-34cd-482c-8f83-9f8439305bcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:45.803601 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a942dfa-34cd-482c-8f83-9f8439305bcf service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:45.803910 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a942dfa-34cd-482c-8f83-9f8439305bcf service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:45.804288 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a942dfa-34cd-482c-8f83-9f8439305bcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:45.806752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9a942dfa-34cd-482c-8f83-9f8439305bcf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:45.904417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29f69d-0e7e-4426-b3f1-0607d0b374e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:45.907867 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29f69d-0e7e-4426-b3f1-0607d0b374e7 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:45.948240 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1347/2700] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:33:45.949567 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29f69d-0e7e-4426-b3f1-0607d0b374e7 service nova] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:45.951232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-0d29f69d-0e7e-4426-b3f1-0607d0b374e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:45.967494 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a942dfa-34cd-482c-8f83-9f8439305bcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:45.970211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29f69d-0e7e-4426-b3f1-0607d0b374e7 service nova] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:45.970441 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a942dfa-34cd-482c-8f83-9f8439305bcf service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:45.976084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb34c1f-6ac8-4216-bf73-62d5ab9cc8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:45.976906 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb34c1f-6ac8-4216-bf73-62d5ab9cc8db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:45.977273 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb34c1f-6ac8-4216-bf73-62d5ab9cc8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:45.978058 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb34c1f-6ac8-4216-bf73-62d5ab9cc8db admin admin] PATH_INFO: `/v3/users/7a3aa23e7ce445649eaab174603fc0a6` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:45.979521 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb34c1f-6ac8-4216-bf73-62d5ab9cc8db admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a3aa23e7ce445649eaab174603fc0a6)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:45.980933 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:45.983849 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb34c1f-6ac8-4216-bf73-62d5ab9cc8db admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:45.994185 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1354/2701] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:33:46.108137 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-ddb34c1f-6ac8-4216-bf73-62d5ab9cc8db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a3aa23e7ce445649eaab174603fc0a6'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:46.124034 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-ddb34c1f-6ac8-4216-bf73-62d5ab9cc8db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a3aa23e7ce445649eaab174603fc0a6'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:46.150733 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-ddb34c1f-6ac8-4216-bf73-62d5ab9cc8db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a3aa23e7ce445649eaab174603fc0a6'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:46.152116 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-ddb34c1f-6ac8-4216-bf73-62d5ab9cc8db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a3aa23e7ce445649eaab174603fc0a6'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:46.168051 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-ddb34c1f-6ac8-4216-bf73-62d5ab9cc8db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a3aa23e7ce445649eaab174603fc0a6'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:46.181474 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1348/2702] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:33:45 2022] DELETE /identity/v3/users/7a3aa23e7ce445649eaab174603fc0a6 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:46.459526 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a942dfa-34cd-482c-8f83-9f8439305bcf service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:46.508544 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a942dfa-34cd-482c-8f83-9f8439305bcf service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:46.628871 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:46.630465 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:46.630854 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:46.631139 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425 admin admin] PATH_INFO: `/v3/projects/933ad15d24bc43b8b6bbe1f1c489924f` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:46.632790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=933ad15d24bc43b8b6bbe1f1c489924f)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:46.635754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:46.706178 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '933ad15d24bc43b8b6bbe1f1c489924f', 'request_id': 'req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:46.718705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '933ad15d24bc43b8b6bbe1f1c489924f', 'request_id': 'req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:46.736616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '933ad15d24bc43b8b6bbe1f1c489924f', 'request_id': 'req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:46.759156 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425 admin admin] The token cache is being invalidate because project 933ad15d24bc43b8b6bbe1f1c489924f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:33:46.759156 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:46.759763 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:46.780048 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1355/2703] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:33:46 2022] DELETE /identity/v3/projects/933ad15d24bc43b8b6bbe1f1c489924f => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:46.815587 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb34c1f-6ac8-4216-bf73-62d5ab9cc8db admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:46.817298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-ddb34c1f-6ac8-4216-bf73-62d5ab9cc8db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:46.835337 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb34c1f-6ac8-4216-bf73-62d5ab9cc8db admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:46.896078 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:46.912865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:46.914846 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:46.915854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:46.916854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66 admin admin] PATH_INFO: `/v3/projects/63336c2d268147e7b8c2bb928273a41d` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:46.918346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63336c2d268147e7b8c2bb928273a41d)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:46.922064 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:46.996383 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63336c2d268147e7b8c2bb928273a41d', 'request_id': 'req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:47.014128 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63336c2d268147e7b8c2bb928273a41d', 'request_id': 'req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:47.029158 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63336c2d268147e7b8c2bb928273a41d', 'request_id': 'req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:47.049653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66 admin admin] The token cache is being invalidate because project 63336c2d268147e7b8c2bb928273a41d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:33:47.050217 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:47.051953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:47.056911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1349/2704] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:33:46 2022] DELETE /identity/v3/projects/63336c2d268147e7b8c2bb928273a41d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:47.059030 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:47.061698 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:47.081163 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1feef-9e44-4a5c-aba5-2bfe25d28425 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:47.188166 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112d81d-801b-4177-ad7d-b56fe804d54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:47.189754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112d81d-801b-4177-ad7d-b56fe804d54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:47.190907 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112d81d-801b-4177-ad7d-b56fe804d54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:47.191887 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112d81d-801b-4177-ad7d-b56fe804d54f admin admin] PATH_INFO: `/v3/users/a1da1de903b64a96bcebed6aeb114bbb` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:47.194564 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112d81d-801b-4177-ad7d-b56fe804d54f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1da1de903b64a96bcebed6aeb114bbb)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:47.197412 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112d81d-801b-4177-ad7d-b56fe804d54f admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:47.268920 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:47.298613 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1112d81d-801b-4177-ad7d-b56fe804d54f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1da1de903b64a96bcebed6aeb114bbb'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:47.323294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1112d81d-801b-4177-ad7d-b56fe804d54f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1da1de903b64a96bcebed6aeb114bbb'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:47.333371 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1112d81d-801b-4177-ad7d-b56fe804d54f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1da1de903b64a96bcebed6aeb114bbb'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:47.334144 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1112d81d-801b-4177-ad7d-b56fe804d54f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1da1de903b64a96bcebed6aeb114bbb'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:47.335385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-1112d81d-801b-4177-ad7d-b56fe804d54f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1da1de903b64a96bcebed6aeb114bbb'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:47.353558 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1356/2705] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:33:47 2022] DELETE /identity/v3/users/a1da1de903b64a96bcebed6aeb114bbb => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:47.900279 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:47.901963 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:47.931556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5babe0-e57a-4615-b1a3-7c4d89e32d66 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:48.058895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646bdf69-aecf-43fc-82ae-27cd2a1eb74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:48.060540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bdf69-aecf-43fc-82ae-27cd2a1eb74b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:48.060899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bdf69-aecf-43fc-82ae-27cd2a1eb74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:48.061244 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bdf69-aecf-43fc-82ae-27cd2a1eb74b admin admin] PATH_INFO: `/v3/projects/62cfd3ed2c51448bb1ff2873cfde7479` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:48.062644 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646bdf69-aecf-43fc-82ae-27cd2a1eb74b admin admin] RBAC: Authorizing `identity:delete_project(project_id=62cfd3ed2c51448bb1ff2873cfde7479)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:48.065483 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646bdf69-aecf-43fc-82ae-27cd2a1eb74b admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:48.144847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-646bdf69-aecf-43fc-82ae-27cd2a1eb74b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62cfd3ed2c51448bb1ff2873cfde7479', 'request_id': 'req-646bdf69-aecf-43fc-82ae-27cd2a1eb74b'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:48.154602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-646bdf69-aecf-43fc-82ae-27cd2a1eb74b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62cfd3ed2c51448bb1ff2873cfde7479', 'request_id': 'req-646bdf69-aecf-43fc-82ae-27cd2a1eb74b'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:48.162393 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-646bdf69-aecf-43fc-82ae-27cd2a1eb74b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62cfd3ed2c51448bb1ff2873cfde7479', 'request_id': 'req-646bdf69-aecf-43fc-82ae-27cd2a1eb74b'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:48.183772 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-646bdf69-aecf-43fc-82ae-27cd2a1eb74b admin admin] The token cache is being invalidate because project 62cfd3ed2c51448bb1ff2873cfde7479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:33:48.184027 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-646bdf69-aecf-43fc-82ae-27cd2a1eb74b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:48.184693 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-646bdf69-aecf-43fc-82ae-27cd2a1eb74b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:48.186266 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1350/2706] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:33:47 2022] DELETE /identity/v3/projects/62cfd3ed2c51448bb1ff2873cfde7479 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:48.193210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076940a-80f5-4706-8eb6-ebca3f863486 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:48.193210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076940a-80f5-4706-8eb6-ebca3f863486 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:48.193210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8076940a-80f5-4706-8eb6-ebca3f863486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:48.214975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-8076940a-80f5-4706-8eb6-ebca3f863486 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:48.216662 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8076940a-80f5-4706-8eb6-ebca3f863486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:48.310186 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1357/2707] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:48.318643 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72c698-921c-40ff-a511-e22324acf955 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:48.318934 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72c698-921c-40ff-a511-e22324acf955 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:48.319209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a72c698-921c-40ff-a511-e22324acf955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:48.344844 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-2a72c698-921c-40ff-a511-e22324acf955 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:48.346061 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-2a72c698-921c-40ff-a511-e22324acf955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:48.377524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1351/2708] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:48.382406 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076940a-80f5-4706-8eb6-ebca3f863486 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:48.395447 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8076940a-80f5-4706-8eb6-ebca3f863486 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:48.400362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a4079-018b-47b6-ad86-62bd772bb760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:48.400995 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a4079-018b-47b6-ad86-62bd772bb760 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:48.401270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a4079-018b-47b6-ad86-62bd772bb760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:48.401542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020a4079-018b-47b6-ad86-62bd772bb760 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:48.402236 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a4079-018b-47b6-ad86-62bd772bb760 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:33:48.402493 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a4079-018b-47b6-ad86-62bd772bb760 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:48.404179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020a4079-018b-47b6-ad86-62bd772bb760 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:48.420068 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1358/2709] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:33:48.432497 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72c698-921c-40ff-a511-e22324acf955 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:48.437362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:48.448533 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a72c698-921c-40ff-a511-e22324acf955 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:48.454718 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec1d8d-3351-41d6-9009-bb31451ad159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:48.455708 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec1d8d-3351-41d6-9009-bb31451ad159 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:48.456102 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec1d8d-3351-41d6-9009-bb31451ad159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:48.456480 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec1d8d-3351-41d6-9009-bb31451ad159 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:48.457275 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec1d8d-3351-41d6-9009-bb31451ad159 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:48.459350 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec1d8d-3351-41d6-9009-bb31451ad159 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:48.496904 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1352/2710] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:33:48 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:48.498115 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a4079-018b-47b6-ad86-62bd772bb760 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:48.508970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020a4079-018b-47b6-ad86-62bd772bb760 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:48.579206 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6e465-6b33-4f04-9f4c-f48c705a09f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:48.579842 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6e465-6b33-4f04-9f4c-f48c705a09f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:48.580200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6e465-6b33-4f04-9f4c-f48c705a09f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:48.580478 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6e465-6b33-4f04-9f4c-f48c705a09f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:48.581127 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6e465-6b33-4f04-9f4c-f48c705a09f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:48.583549 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6e465-6b33-4f04-9f4c-f48c705a09f8 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:48.614278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1359/2711] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:33:48 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:48.618762 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec1d8d-3351-41d6-9009-bb31451ad159 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:48.630542 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec1d8d-3351-41d6-9009-bb31451ad159 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:48.637225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6fc36-05d7-4e66-996d-adbc248b6bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:48.638253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6fc36-05d7-4e66-996d-adbc248b6bbd admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:48.638693 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6fc36-05d7-4e66-996d-adbc248b6bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:48.639158 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6fc36-05d7-4e66-996d-adbc248b6bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:48.639978 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd6fc36-05d7-4e66-996d-adbc248b6bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:48.641880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd6fc36-05d7-4e66-996d-adbc248b6bbd admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:48.655961 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1353/2712] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:48.660338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6e465-6b33-4f04-9f4c-f48c705a09f8 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:48.671902 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6e465-6b33-4f04-9f4c-f48c705a09f8 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:48.678356 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92672c1-2207-475e-87c8-9f745185049e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:48.679450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92672c1-2207-475e-87c8-9f745185049e admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:48.680042 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92672c1-2207-475e-87c8-9f745185049e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:48.680328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92672c1-2207-475e-87c8-9f745185049e admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:48.681316 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92672c1-2207-475e-87c8-9f745185049e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:48.683148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92672c1-2207-475e-87c8-9f745185049e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:48.703066 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1360/2713] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:48.707175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6fc36-05d7-4e66-996d-adbc248b6bbd admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:48.719594 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6fc36-05d7-4e66-996d-adbc248b6bbd admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:48.725270 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f21b9-cb61-45db-a62c-cf15770c8d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:48.726558 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f21b9-cb61-45db-a62c-cf15770c8d3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:48.727344 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f21b9-cb61-45db-a62c-cf15770c8d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:48.730819 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f21b9-cb61-45db-a62c-cf15770c8d3f admin admin] PATH_INFO: `/v3/projects/e8c36dfb510c4eb5889eca0ada6f2ec9/users/0848463840c444ebab3c67bf0529ef3e/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:48.754651 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f21b9-cb61-45db-a62c-cf15770c8d3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8c36dfb510c4eb5889eca0ada6f2ec9, user_id=0848463840c444ebab3c67bf0529ef3e, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:48.757675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f21b9-cb61-45db-a62c-cf15770c8d3f admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:48.775228 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1354/2714] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:48 2022] PUT /identity/v3/projects/e8c36dfb510c4eb5889eca0ada6f2ec9/users/0848463840c444ebab3c67bf0529ef3e/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:48.785829 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc1b20-6c1f-4597-aaa8-facf78c6d152 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:48.786303 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc1b20-6c1f-4597-aaa8-facf78c6d152 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:48.786664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bc1b20-6c1f-4597-aaa8-facf78c6d152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:48.822508 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-e5bc1b20-6c1f-4597-aaa8-facf78c6d152 None None] MFA Rules not processed for user `0848463840c444ebab3c67bf0529ef3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:48.823776 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e5bc1b20-6c1f-4597-aaa8-facf78c6d152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:48.829798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:48.933247 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1361/2715] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:48.939075 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f21b9-cb61-45db-a62c-cf15770c8d3f admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:49.020995 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f21b9-cb61-45db-a62c-cf15770c8d3f admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:49.026322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874c6f0-914c-4c59-a76f-96796e250ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:49.026967 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874c6f0-914c-4c59-a76f-96796e250ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:49.027231 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874c6f0-914c-4c59-a76f-96796e250ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:49.027680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0874c6f0-914c-4c59-a76f-96796e250ed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:49.028709 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874c6f0-914c-4c59-a76f-96796e250ed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:49.030694 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0874c6f0-914c-4c59-a76f-96796e250ed7 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:49.074309 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1355/2716] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:33:48 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:49.076223 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc1b20-6c1f-4597-aaa8-facf78c6d152 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:49.089881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bc1b20-6c1f-4597-aaa8-facf78c6d152 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:49.173521 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd7598-db48-4796-8b96-85a383c99501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:49.174221 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd7598-db48-4796-8b96-85a383c99501 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:49.174796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd7598-db48-4796-8b96-85a383c99501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:49.174796 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd7598-db48-4796-8b96-85a383c99501 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:49.175429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd7598-db48-4796-8b96-85a383c99501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:49.177763 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acd7598-db48-4796-8b96-85a383c99501 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:49.223164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1362/2717] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:33:49 2022] POST /identity/v3/users => generated 297 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:49.227829 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874c6f0-914c-4c59-a76f-96796e250ed7 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:49.244603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0874c6f0-914c-4c59-a76f-96796e250ed7 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:49.259051 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a386d-c946-4e24-854f-8d07f4ded3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:49.260277 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a386d-c946-4e24-854f-8d07f4ded3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:49.260768 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a386d-c946-4e24-854f-8d07f4ded3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:49.261471 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a386d-c946-4e24-854f-8d07f4ded3d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:49.263622 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a386d-c946-4e24-854f-8d07f4ded3d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:49.267847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a386d-c946-4e24-854f-8d07f4ded3d1 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:49.303426 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1356/2718] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:49.305487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd7598-db48-4796-8b96-85a383c99501 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:49.322347 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd7598-db48-4796-8b96-85a383c99501 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:49.333486 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d3df7-a9fb-47a5-8874-e3b9fa415a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:49.334729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d3df7-a9fb-47a5-8874-e3b9fa415a32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:49.335214 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d3df7-a9fb-47a5-8874-e3b9fa415a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:49.335711 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d3df7-a9fb-47a5-8874-e3b9fa415a32 admin admin] PATH_INFO: `/v3/projects/d97f1a52c13d48e6b9b5258fdc19c6b6/users/32b182293f814fb08b294554092936ef/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:49.366677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d3df7-a9fb-47a5-8874-e3b9fa415a32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d97f1a52c13d48e6b9b5258fdc19c6b6, user_id=32b182293f814fb08b294554092936ef, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:49.371130 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d3df7-a9fb-47a5-8874-e3b9fa415a32 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:49.395696 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1363/2719] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:49 2022] PUT /identity/v3/projects/d97f1a52c13d48e6b9b5258fdc19c6b6/users/32b182293f814fb08b294554092936ef/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:49.400894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a386d-c946-4e24-854f-8d07f4ded3d1 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:49.413787 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a386d-c946-4e24-854f-8d07f4ded3d1 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:49.520200 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27f25f-c8d7-451d-bc47-8625424d7613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:49.521144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27f25f-c8d7-451d-bc47-8625424d7613 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:49.521606 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27f25f-c8d7-451d-bc47-8625424d7613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:49.522132 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27f25f-c8d7-451d-bc47-8625424d7613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:49.523212 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27f25f-c8d7-451d-bc47-8625424d7613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:49.525348 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27f25f-c8d7-451d-bc47-8625424d7613 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:49.542775 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1357/2720] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:49.548191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d3df7-a9fb-47a5-8874-e3b9fa415a32 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:49.560490 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d3df7-a9fb-47a5-8874-e3b9fa415a32 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:49.565815 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778bf77-48b9-42ca-82a3-270f8a24cf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:49.566500 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778bf77-48b9-42ca-82a3-270f8a24cf0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:49.566770 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778bf77-48b9-42ca-82a3-270f8a24cf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:49.567041 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778bf77-48b9-42ca-82a3-270f8a24cf0e admin admin] PATH_INFO: `/v3/projects/d97f1a52c13d48e6b9b5258fdc19c6b6/users/32b182293f814fb08b294554092936ef/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:49.569280 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778bf77-48b9-42ca-82a3-270f8a24cf0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d97f1a52c13d48e6b9b5258fdc19c6b6, user_id=32b182293f814fb08b294554092936ef, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:49.572038 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778bf77-48b9-42ca-82a3-270f8a24cf0e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:49.584399 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1364/2721] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:49 2022] PUT /identity/v3/projects/d97f1a52c13d48e6b9b5258fdc19c6b6/users/32b182293f814fb08b294554092936ef/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:49.596611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac80df1-d1fa-4484-8724-9684f72b7919 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:49.597318 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac80df1-d1fa-4484-8724-9684f72b7919 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:49.597881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac80df1-d1fa-4484-8724-9684f72b7919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:49.646574 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-fac80df1-d1fa-4484-8724-9684f72b7919 None None] MFA Rules not processed for user `32b182293f814fb08b294554092936ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:49.647906 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-fac80df1-d1fa-4484-8724-9684f72b7919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:49.652904 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:49.742674 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1358/2722] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:49.761113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778bf77-48b9-42ca-82a3-270f8a24cf0e admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:49.764144 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-2778bf77-48b9-42ca-82a3-270f8a24cf0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:49.885387 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778bf77-48b9-42ca-82a3-270f8a24cf0e admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:49.889897 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae4145-36e9-4d6b-8602-7f7776cfa01c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:49.895057 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae4145-36e9-4d6b-8602-7f7776cfa01c service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:49.895057 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae4145-36e9-4d6b-8602-7f7776cfa01c service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:49.895057 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae4145-36e9-4d6b-8602-7f7776cfa01c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:49.906125 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae4145-36e9-4d6b-8602-7f7776cfa01c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:49.907829 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae4145-36e9-4d6b-8602-7f7776cfa01c service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:49.914234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:49.921371 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1365/2723] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:33:50.556857 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac80df1-d1fa-4484-8724-9684f72b7919 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:50.642631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac80df1-d1fa-4484-8724-9684f72b7919 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:50.647733 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699db77-9ce0-4078-88a1-465ab3e012ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:50.648677 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699db77-9ce0-4078-88a1-465ab3e012ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:50.649055 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699db77-9ce0-4078-88a1-465ab3e012ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:50.649430 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3699db77-9ce0-4078-88a1-465ab3e012ad admin admin] PATH_INFO: `/v3/users/0848463840c444ebab3c67bf0529ef3e` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:50.650862 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699db77-9ce0-4078-88a1-465ab3e012ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=0848463840c444ebab3c67bf0529ef3e)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:50.653172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3699db77-9ce0-4078-88a1-465ab3e012ad admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:50.717797 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3699db77-9ce0-4078-88a1-465ab3e012ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0848463840c444ebab3c67bf0529ef3e'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:50.727597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3699db77-9ce0-4078-88a1-465ab3e012ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0848463840c444ebab3c67bf0529ef3e'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:50.743305 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3699db77-9ce0-4078-88a1-465ab3e012ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0848463840c444ebab3c67bf0529ef3e'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:50.744363 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3699db77-9ce0-4078-88a1-465ab3e012ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0848463840c444ebab3c67bf0529ef3e'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:50.753943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-3699db77-9ce0-4078-88a1-465ab3e012ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0848463840c444ebab3c67bf0529ef3e'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:50.764122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1359/2724] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:33:50 2022] DELETE /identity/v3/users/0848463840c444ebab3c67bf0529ef3e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:50.770203 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2091e33-177c-424c-a182-6ad5878b3582 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:50.770806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2091e33-177c-424c-a182-6ad5878b3582 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:50.770958 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2091e33-177c-424c-a182-6ad5878b3582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:50.802717 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-d2091e33-177c-424c-a182-6ad5878b3582 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:50.804561 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d2091e33-177c-424c-a182-6ad5878b3582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:50.897155 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1366/2725] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:33:50.902623 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699db77-9ce0-4078-88a1-465ab3e012ad admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:50.904757 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3699db77-9ce0-4078-88a1-465ab3e012ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:50.932567 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:50.983057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3699db77-9ce0-4078-88a1-465ab3e012ad admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:50.987086 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509b3d4-14f8-4cea-8a8a-eaf89f9f31ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:50.987932 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509b3d4-14f8-4cea-8a8a-eaf89f9f31ec service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:50.988256 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509b3d4-14f8-4cea-8a8a-eaf89f9f31ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:50.988520 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509b3d4-14f8-4cea-8a8a-eaf89f9f31ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:51.000226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509b3d4-14f8-4cea-8a8a-eaf89f9f31ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:51.001871 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509b3d4-14f8-4cea-8a8a-eaf89f9f31ec service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:51.019030 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1360/2726] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:33:51.376139 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2091e33-177c-424c-a182-6ad5878b3582 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:51.378620 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d2091e33-177c-424c-a182-6ad5878b3582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:51.399247 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2091e33-177c-424c-a182-6ad5878b3582 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:51.408208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf2c52-ea53-4288-91f7-67647d4d1838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:51.410468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf2c52-ea53-4288-91f7-67647d4d1838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:51.411226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf2c52-ea53-4288-91f7-67647d4d1838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:51.411729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf2c52-ea53-4288-91f7-67647d4d1838 admin admin] PATH_INFO: `/v3/projects/e8c36dfb510c4eb5889eca0ada6f2ec9` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:51.414017 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf2c52-ea53-4288-91f7-67647d4d1838 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8c36dfb510c4eb5889eca0ada6f2ec9)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:51.417265 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf2c52-ea53-4288-91f7-67647d4d1838 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:51.497819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a0cf2c52-ea53-4288-91f7-67647d4d1838 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8c36dfb510c4eb5889eca0ada6f2ec9', 'request_id': 'req-a0cf2c52-ea53-4288-91f7-67647d4d1838'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:51.505123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a0cf2c52-ea53-4288-91f7-67647d4d1838 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8c36dfb510c4eb5889eca0ada6f2ec9', 'request_id': 'req-a0cf2c52-ea53-4288-91f7-67647d4d1838'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:51.514070 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a0cf2c52-ea53-4288-91f7-67647d4d1838 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8c36dfb510c4eb5889eca0ada6f2ec9', 'request_id': 'req-a0cf2c52-ea53-4288-91f7-67647d4d1838'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:51.526325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a0cf2c52-ea53-4288-91f7-67647d4d1838 admin admin] The token cache is being invalidate because project e8c36dfb510c4eb5889eca0ada6f2ec9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:33:51.526637 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a0cf2c52-ea53-4288-91f7-67647d4d1838 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:51.527405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-a0cf2c52-ea53-4288-91f7-67647d4d1838 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:51.529281 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1367/2727] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:33:51 2022] DELETE /identity/v3/projects/e8c36dfb510c4eb5889eca0ada6f2ec9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:51.534313 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509b3d4-14f8-4cea-8a8a-eaf89f9f31ec service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:51.547675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509b3d4-14f8-4cea-8a8a-eaf89f9f31ec service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:51.622199 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e53ca-5840-4b87-8444-85a009b66673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:51.622981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e53ca-5840-4b87-8444-85a009b66673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:51.623113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e53ca-5840-4b87-8444-85a009b66673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:51.623405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e53ca-5840-4b87-8444-85a009b66673 admin admin] PATH_INFO: `/v3/users/32b182293f814fb08b294554092936ef` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:51.624751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e53ca-5840-4b87-8444-85a009b66673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32b182293f814fb08b294554092936ef)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:51.627035 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e53ca-5840-4b87-8444-85a009b66673 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:51.695582 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-566e53ca-5840-4b87-8444-85a009b66673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32b182293f814fb08b294554092936ef'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:51.705014 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-566e53ca-5840-4b87-8444-85a009b66673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32b182293f814fb08b294554092936ef'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:51.726021 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-566e53ca-5840-4b87-8444-85a009b66673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32b182293f814fb08b294554092936ef'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:51.726866 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-566e53ca-5840-4b87-8444-85a009b66673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32b182293f814fb08b294554092936ef'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:51.737896 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-566e53ca-5840-4b87-8444-85a009b66673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32b182293f814fb08b294554092936ef'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:51.746144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1361/2728] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:33:51 2022] DELETE /identity/v3/users/32b182293f814fb08b294554092936ef => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:52.205234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf2c52-ea53-4288-91f7-67647d4d1838 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:52.207617 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-a0cf2c52-ea53-4288-91f7-67647d4d1838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:52.237689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf2c52-ea53-4288-91f7-67647d4d1838 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:52.313536 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328d2ff-699a-4847-93d9-954a2f912e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:52.314269 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328d2ff-699a-4847-93d9-954a2f912e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:52.314633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328d2ff-699a-4847-93d9-954a2f912e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:52.315038 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328d2ff-699a-4847-93d9-954a2f912e61 admin admin] PATH_INFO: `/v3/projects/d97f1a52c13d48e6b9b5258fdc19c6b6` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:52.316839 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328d2ff-699a-4847-93d9-954a2f912e61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d97f1a52c13d48e6b9b5258fdc19c6b6)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:52.319013 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328d2ff-699a-4847-93d9-954a2f912e61 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:52.357964 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:52.381459 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5328d2ff-699a-4847-93d9-954a2f912e61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd97f1a52c13d48e6b9b5258fdc19c6b6', 'request_id': 'req-5328d2ff-699a-4847-93d9-954a2f912e61'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:52.388487 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5328d2ff-699a-4847-93d9-954a2f912e61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd97f1a52c13d48e6b9b5258fdc19c6b6', 'request_id': 'req-5328d2ff-699a-4847-93d9-954a2f912e61'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:52.397850 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5328d2ff-699a-4847-93d9-954a2f912e61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd97f1a52c13d48e6b9b5258fdc19c6b6', 'request_id': 'req-5328d2ff-699a-4847-93d9-954a2f912e61'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:52.413579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5328d2ff-699a-4847-93d9-954a2f912e61 admin admin] The token cache is being invalidate because project d97f1a52c13d48e6b9b5258fdc19c6b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:33:52.413893 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5328d2ff-699a-4847-93d9-954a2f912e61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:52.414828 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-5328d2ff-699a-4847-93d9-954a2f912e61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:52.416951 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1368/2729] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:33:52 2022] DELETE /identity/v3/projects/d97f1a52c13d48e6b9b5258fdc19c6b6 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:52.423951 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dac4b-4417-49c3-90a3-9a6d95c23d77 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:52.424378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dac4b-4417-49c3-90a3-9a6d95c23d77 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:52.424833 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dac4b-4417-49c3-90a3-9a6d95c23d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:52.447715 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-a95dac4b-4417-49c3-90a3-9a6d95c23d77 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:52.449257 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a95dac4b-4417-49c3-90a3-9a6d95c23d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:52.495956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:52.526352 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1362/2730] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:33:52.631075 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78c5b1-f1be-4f88-8bb3-bae552d4dd52 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:52.631320 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78c5b1-f1be-4f88-8bb3-bae552d4dd52 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:52.631648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a78c5b1-f1be-4f88-8bb3-bae552d4dd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:52.656953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-3a78c5b1-f1be-4f88-8bb3-bae552d4dd52 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:52.658220 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-3a78c5b1-f1be-4f88-8bb3-bae552d4dd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:52.668819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1369/2731] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:52.674304 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dac4b-4417-49c3-90a3-9a6d95c23d77 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:52.687595 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dac4b-4417-49c3-90a3-9a6d95c23d77 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:52.693691 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534af09-59e9-4023-a21f-602cd5ab4338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:52.694339 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534af09-59e9-4023-a21f-602cd5ab4338 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:52.694620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534af09-59e9-4023-a21f-602cd5ab4338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:52.694884 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534af09-59e9-4023-a21f-602cd5ab4338 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:52.695562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534af09-59e9-4023-a21f-602cd5ab4338 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:33:52.695861 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534af09-59e9-4023-a21f-602cd5ab4338 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:52.697917 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534af09-59e9-4023-a21f-602cd5ab4338 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:52.712282 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1363/2732] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:33:52.717560 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78c5b1-f1be-4f88-8bb3-bae552d4dd52 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:52.730952 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a78c5b1-f1be-4f88-8bb3-bae552d4dd52 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:52.736996 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca67ae4-2364-4287-8988-0af7ccbc6aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:52.737813 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca67ae4-2364-4287-8988-0af7ccbc6aa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:52.738094 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca67ae4-2364-4287-8988-0af7ccbc6aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:52.738369 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca67ae4-2364-4287-8988-0af7ccbc6aa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:52.739047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca67ae4-2364-4287-8988-0af7ccbc6aa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:52.741223 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca67ae4-2364-4287-8988-0af7ccbc6aa4 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:52.776076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1370/2733] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:33:52 2022] POST /identity/v3/projects => generated 361 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:52.782346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534af09-59e9-4023-a21f-602cd5ab4338 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:52.794438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534af09-59e9-4023-a21f-602cd5ab4338 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:52.869179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce462cf-9ed5-44c4-8e39-e742bef68c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:52.869841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce462cf-9ed5-44c4-8e39-e742bef68c05 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:52.870109 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce462cf-9ed5-44c4-8e39-e742bef68c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:52.870392 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce462cf-9ed5-44c4-8e39-e742bef68c05 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:52.871037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce462cf-9ed5-44c4-8e39-e742bef68c05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:52.873144 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce462cf-9ed5-44c4-8e39-e742bef68c05 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:52.899321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1364/2734] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:33:52 2022] POST /identity/v3/users => generated 281 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:52.906586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca67ae4-2364-4287-8988-0af7ccbc6aa4 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:52.925054 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca67ae4-2364-4287-8988-0af7ccbc6aa4 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:52.930949 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93755659-66d6-4e76-a7c0-6571ab105c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:52.931822 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93755659-66d6-4e76-a7c0-6571ab105c70 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:52.932183 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93755659-66d6-4e76-a7c0-6571ab105c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:52.932503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93755659-66d6-4e76-a7c0-6571ab105c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:52.933340 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93755659-66d6-4e76-a7c0-6571ab105c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:52.935061 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93755659-66d6-4e76-a7c0-6571ab105c70 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:52.947898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1371/2735] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:33:52.952842 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce462cf-9ed5-44c4-8e39-e742bef68c05 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:52.968190 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce462cf-9ed5-44c4-8e39-e742bef68c05 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:52.974613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c26f8b-107a-42b7-83fd-6d9ff07323b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:52.975330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c26f8b-107a-42b7-83fd-6d9ff07323b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:52.975606 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c26f8b-107a-42b7-83fd-6d9ff07323b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:52.975881 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c26f8b-107a-42b7-83fd-6d9ff07323b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:52.976619 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c26f8b-107a-42b7-83fd-6d9ff07323b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:52.978784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c26f8b-107a-42b7-83fd-6d9ff07323b5 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:52.992619 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1365/2736] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:52.997450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93755659-66d6-4e76-a7c0-6571ab105c70 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:53.009608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93755659-66d6-4e76-a7c0-6571ab105c70 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:53.016444 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05a713-e9af-4783-94b7-621b75f519ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:53.017329 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05a713-e9af-4783-94b7-621b75f519ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:53.017825 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05a713-e9af-4783-94b7-621b75f519ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:53.018751 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05a713-e9af-4783-94b7-621b75f519ab admin admin] PATH_INFO: `/v3/projects/15d26b479ba146f98e04add890123c0d/users/0c372662af4e4dd789c63c70fbbc12b2/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:53.050754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05a713-e9af-4783-94b7-621b75f519ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=15d26b479ba146f98e04add890123c0d, user_id=0c372662af4e4dd789c63c70fbbc12b2, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:53.056112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05a713-e9af-4783-94b7-621b75f519ab admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:53.076168 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1372/2737] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:52 2022] PUT /identity/v3/projects/15d26b479ba146f98e04add890123c0d/users/0c372662af4e4dd789c63c70fbbc12b2/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:33:53.087074 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9cd0f6-ebd5-47f8-a498-e091c8a414e1 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:53.087370 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9cd0f6-ebd5-47f8-a498-e091c8a414e1 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:53.087747 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9cd0f6-ebd5-47f8-a498-e091c8a414e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:53.124890 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-fe9cd0f6-ebd5-47f8-a498-e091c8a414e1 None None] MFA Rules not processed for user `0c372662af4e4dd789c63c70fbbc12b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:53.126158 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-fe9cd0f6-ebd5-47f8-a498-e091c8a414e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:53.202222 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1366/2738] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:53.242893 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05a713-e9af-4783-94b7-621b75f519ab admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:53.245189 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-fa05a713-e9af-4783-94b7-621b75f519ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:53.317606 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05a713-e9af-4783-94b7-621b75f519ab admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:53.321672 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a817a-c8b5-4e0e-92d6-6679eb64b555 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:53.322322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a817a-c8b5-4e0e-92d6-6679eb64b555 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:53.322612 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a817a-c8b5-4e0e-92d6-6679eb64b555 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:53.322889 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a817a-c8b5-4e0e-92d6-6679eb64b555 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:53.337294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a817a-c8b5-4e0e-92d6-6679eb64b555 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:53.339207 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a817a-c8b5-4e0e-92d6-6679eb64b555 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:53.350550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1373/2739] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:33:53.368287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9cd0f6-ebd5-47f8-a498-e091c8a414e1 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:53.417424 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:53.447316 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9cd0f6-ebd5-47f8-a498-e091c8a414e1 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:53.453712 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37058b5f-6a84-4888-bb6f-e68f1966a69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:53.454326 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37058b5f-6a84-4888-bb6f-e68f1966a69f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:53.454603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37058b5f-6a84-4888-bb6f-e68f1966a69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:53.454876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37058b5f-6a84-4888-bb6f-e68f1966a69f admin admin] PATH_INFO: `/v3/users/0c372662af4e4dd789c63c70fbbc12b2` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:53.456161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37058b5f-6a84-4888-bb6f-e68f1966a69f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c372662af4e4dd789c63c70fbbc12b2)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:53.458221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37058b5f-6a84-4888-bb6f-e68f1966a69f admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:53.520419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-37058b5f-6a84-4888-bb6f-e68f1966a69f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c372662af4e4dd789c63c70fbbc12b2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:53.529285 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-37058b5f-6a84-4888-bb6f-e68f1966a69f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c372662af4e4dd789c63c70fbbc12b2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:53.545349 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-37058b5f-6a84-4888-bb6f-e68f1966a69f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c372662af4e4dd789c63c70fbbc12b2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:53.546675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-37058b5f-6a84-4888-bb6f-e68f1966a69f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c372662af4e4dd789c63c70fbbc12b2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:53.559004 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-37058b5f-6a84-4888-bb6f-e68f1966a69f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c372662af4e4dd789c63c70fbbc12b2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:53.568961 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1367/2740] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:33:53 2022] DELETE /identity/v3/users/0c372662af4e4dd789c63c70fbbc12b2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:53.574934 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8575a95-4f98-4486-b61e-5e3b27ead70c None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:53.575243 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8575a95-4f98-4486-b61e-5e3b27ead70c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:53.575596 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8575a95-4f98-4486-b61e-5e3b27ead70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:53.598749 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-e8575a95-4f98-4486-b61e-5e3b27ead70c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:53.600016 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e8575a95-4f98-4486-b61e-5e3b27ead70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:53.665734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:53.691585 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1374/2741] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:53.711892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37058b5f-6a84-4888-bb6f-e68f1966a69f admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:53.714168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-37058b5f-6a84-4888-bb6f-e68f1966a69f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:53.800204 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37058b5f-6a84-4888-bb6f-e68f1966a69f admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:53.806122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06576da5-3c59-44a1-aa54-0bc2eef314a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:53.807174 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576da5-3c59-44a1-aa54-0bc2eef314a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:53.807678 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576da5-3c59-44a1-aa54-0bc2eef314a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:53.808248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576da5-3c59-44a1-aa54-0bc2eef314a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:53.827355 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06576da5-3c59-44a1-aa54-0bc2eef314a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:53.830927 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06576da5-3c59-44a1-aa54-0bc2eef314a3 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:53.851372 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1368/2742] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:33:54.224643 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8575a95-4f98-4486-b61e-5e3b27ead70c None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:54.227294 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e8575a95-4f98-4486-b61e-5e3b27ead70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:54.245607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8575a95-4f98-4486-b61e-5e3b27ead70c None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:54.252050 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111ac4f-1765-497a-aedf-2e2e2f24bb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:54.252707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111ac4f-1765-497a-aedf-2e2e2f24bb47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:54.252986 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111ac4f-1765-497a-aedf-2e2e2f24bb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:54.253290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111ac4f-1765-497a-aedf-2e2e2f24bb47 admin admin] PATH_INFO: `/v3/projects/15d26b479ba146f98e04add890123c0d` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:54.254605 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111ac4f-1765-497a-aedf-2e2e2f24bb47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15d26b479ba146f98e04add890123c0d)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:54.256623 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111ac4f-1765-497a-aedf-2e2e2f24bb47 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:54.317365 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-b111ac4f-1765-497a-aedf-2e2e2f24bb47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15d26b479ba146f98e04add890123c0d', 'request_id': 'req-b111ac4f-1765-497a-aedf-2e2e2f24bb47'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:54.325700 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-b111ac4f-1765-497a-aedf-2e2e2f24bb47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15d26b479ba146f98e04add890123c0d', 'request_id': 'req-b111ac4f-1765-497a-aedf-2e2e2f24bb47'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:54.335586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-b111ac4f-1765-497a-aedf-2e2e2f24bb47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15d26b479ba146f98e04add890123c0d', 'request_id': 'req-b111ac4f-1765-497a-aedf-2e2e2f24bb47'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:54.352438 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-b111ac4f-1765-497a-aedf-2e2e2f24bb47 admin admin] The token cache is being invalidate because project 15d26b479ba146f98e04add890123c0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:33:54.352952 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-b111ac4f-1765-497a-aedf-2e2e2f24bb47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:54.354150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-b111ac4f-1765-497a-aedf-2e2e2f24bb47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:33:54.357054 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1375/2743] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:33:54 2022] DELETE /identity/v3/projects/15d26b479ba146f98e04add890123c0d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:33:54.363468 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629103f-1f10-48fa-88c4-d0f021d3ed46 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:54.363788 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629103f-1f10-48fa-88c4-d0f021d3ed46 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:54.364037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7629103f-1f10-48fa-88c4-d0f021d3ed46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:54.385502 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-7629103f-1f10-48fa-88c4-d0f021d3ed46 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:54.386771 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-7629103f-1f10-48fa-88c4-d0f021d3ed46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:54.461325 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:33:54.476976 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1369/2744] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:33:54.489229 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c65e1-7936-460c-a3be-037a3d046ae3 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:54.489510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c65e1-7936-460c-a3be-037a3d046ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:54.489779 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c65e1-7936-460c-a3be-037a3d046ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:54.513025 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-5f5c65e1-7936-460c-a3be-037a3d046ae3 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:54.514322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-5f5c65e1-7936-460c-a3be-037a3d046ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:54.524789 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1376/2745] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:33:54.532214 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629103f-1f10-48fa-88c4-d0f021d3ed46 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:54.550996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7629103f-1f10-48fa-88c4-d0f021d3ed46 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:54.558943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8952d87-4caf-4074-bcb1-ce319e3ddad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:54.560020 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8952d87-4caf-4074-bcb1-ce319e3ddad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:54.560504 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8952d87-4caf-4074-bcb1-ce319e3ddad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:54.560981 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8952d87-4caf-4074-bcb1-ce319e3ddad9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:54.562172 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8952d87-4caf-4074-bcb1-ce319e3ddad9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:33:54.562702 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8952d87-4caf-4074-bcb1-ce319e3ddad9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:54.565945 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8952d87-4caf-4074-bcb1-ce319e3ddad9 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:54.583802 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1370/2746] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:33:54.591150 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c65e1-7936-460c-a3be-037a3d046ae3 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:54.605377 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c65e1-7936-460c-a3be-037a3d046ae3 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:54.611641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dc8c8-af2a-4556-b356-30af72d76fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:54.612370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dc8c8-af2a-4556-b356-30af72d76fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:54.612643 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dc8c8-af2a-4556-b356-30af72d76fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:54.612899 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dc8c8-af2a-4556-b356-30af72d76fc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:54.613785 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3dc8c8-af2a-4556-b356-30af72d76fc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:54.615664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3dc8c8-af2a-4556-b356-30af72d76fc2 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:54.655556 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1377/2747] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:33:54 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:54.660327 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8952d87-4caf-4074-bcb1-ce319e3ddad9 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:54.672663 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8952d87-4caf-4074-bcb1-ce319e3ddad9 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:54.750243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f1fc0-7091-45ed-9c37-36836b4ced4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:54.751048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f1fc0-7091-45ed-9c37-36836b4ced4d admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:54.751221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f1fc0-7091-45ed-9c37-36836b4ced4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:54.751371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f1fc0-7091-45ed-9c37-36836b4ced4d admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:54.751972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f1fc0-7091-45ed-9c37-36836b4ced4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:54.754253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f1fc0-7091-45ed-9c37-36836b4ced4d admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:54.785338 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1371/2748] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:33:54 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:33:54.793306 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dc8c8-af2a-4556-b356-30af72d76fc2 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:54.806384 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dc8c8-af2a-4556-b356-30af72d76fc2 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:54.812244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8a573-bccc-4a64-a154-01fbf4ee6278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:54.813069 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8a573-bccc-4a64-a154-01fbf4ee6278 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:54.813545 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8a573-bccc-4a64-a154-01fbf4ee6278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:54.813877 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8a573-bccc-4a64-a154-01fbf4ee6278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:54.814689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8a573-bccc-4a64-a154-01fbf4ee6278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:54.816454 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8a573-bccc-4a64-a154-01fbf4ee6278 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:54.839118 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1378/2749] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:54.843759 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f1fc0-7091-45ed-9c37-36836b4ced4d admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:54.856984 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f1fc0-7091-45ed-9c37-36836b4ced4d admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:54.865015 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24459fcc-947b-4c70-8756-a14e01dc49b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:54.865015 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24459fcc-947b-4c70-8756-a14e01dc49b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:54.865015 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24459fcc-947b-4c70-8756-a14e01dc49b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:54.865015 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24459fcc-947b-4c70-8756-a14e01dc49b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:54.865905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24459fcc-947b-4c70-8756-a14e01dc49b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:54.868289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24459fcc-947b-4c70-8756-a14e01dc49b6 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:54.883289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1372/2750] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:33:54.888345 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8a573-bccc-4a64-a154-01fbf4ee6278 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:54.900183 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8a573-bccc-4a64-a154-01fbf4ee6278 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:54.906058 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbcb11-30c0-405f-9e1f-1236ca21a7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:54.906904 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbcb11-30c0-405f-9e1f-1236ca21a7bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:54.907285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbcb11-30c0-405f-9e1f-1236ca21a7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:54.907661 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbcb11-30c0-405f-9e1f-1236ca21a7bc admin admin] PATH_INFO: `/v3/projects/1b16920c2e384c9ca20252917204b8ab/users/523936e9a8ca49b9805fa110a1841d64/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:54.927026 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbcb11-30c0-405f-9e1f-1236ca21a7bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b16920c2e384c9ca20252917204b8ab, user_id=523936e9a8ca49b9805fa110a1841d64, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:54.930076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbcb11-30c0-405f-9e1f-1236ca21a7bc admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:54.949728 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1379/2751] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:33:54 2022] PUT /identity/v3/projects/1b16920c2e384c9ca20252917204b8ab/users/523936e9a8ca49b9805fa110a1841d64/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:33:54.956123 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3a761-cb78-481c-9d42-2b1e70a2a307 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:54.956698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3a761-cb78-481c-9d42-2b1e70a2a307 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:54.957184 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3a761-cb78-481c-9d42-2b1e70a2a307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:54.990684 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-f9f3a761-cb78-481c-9d42-2b1e70a2a307 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:33:54.993321 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-f9f3a761-cb78-481c-9d42-2b1e70a2a307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:55.082094 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1373/2752] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:33:55.093345 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbcb11-30c0-405f-9e1f-1236ca21a7bc admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:33:55.095510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-dedbcb11-30c0-405f-9e1f-1236ca21a7bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:33:55.166247 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbcb11-30c0-405f-9e1f-1236ca21a7bc admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:33:55.170687 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a33a6-fe27-4377-8b30-ae3857cf9ab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:33:55.171328 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a33a6-fe27-4377-8b30-ae3857cf9ab7 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:33:55.171607 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a33a6-fe27-4377-8b30-ae3857cf9ab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:33:55.171873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a33a6-fe27-4377-8b30-ae3857cf9ab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:33:55.182855 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a33a6-fe27-4377-8b30-ae3857cf9ab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:33:55.184406 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a33a6-fe27-4377-8b30-ae3857cf9ab7 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:33:55.195906 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1380/2753] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:33:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:34:00.819922 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef409d8-d501-481f-9e90-773c1d457962 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:00.820859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef409d8-d501-481f-9e90-773c1d457962 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:00.820859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef409d8-d501-481f-9e90-773c1d457962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:00.868746 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-aef409d8-d501-481f-9e90-773c1d457962 None None] MFA Rules not processed for user `523936e9a8ca49b9805fa110a1841d64`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:00.870869 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-aef409d8-d501-481f-9e90-773c1d457962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:00.998806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:01.019501 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1374/2754] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:34:01.025362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a33a6-fe27-4377-8b30-ae3857cf9ab7 service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:01.051381 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a33a6-fe27-4377-8b30-ae3857cf9ab7 service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:01.058224 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316fe3a-7f5d-4f97-902a-c46907cd25fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:01.059079 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316fe3a-7f5d-4f97-902a-c46907cd25fe admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:01.059509 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316fe3a-7f5d-4f97-902a-c46907cd25fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:01.059926 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316fe3a-7f5d-4f97-902a-c46907cd25fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:01.061011 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316fe3a-7f5d-4f97-902a-c46907cd25fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:01.063748 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316fe3a-7f5d-4f97-902a-c46907cd25fe admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:01.083505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:01.104760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1381/2755] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:34:01 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:34:01.109582 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef409d8-d501-481f-9e90-773c1d457962 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:01.123114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef409d8-d501-481f-9e90-773c1d457962 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:01.230242 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c877f35-8c96-408a-9ebc-8ff6f2a38b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:01.230899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c877f35-8c96-408a-9ebc-8ff6f2a38b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:01.231173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c877f35-8c96-408a-9ebc-8ff6f2a38b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:01.231453 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c877f35-8c96-408a-9ebc-8ff6f2a38b50 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:01.232160 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c877f35-8c96-408a-9ebc-8ff6f2a38b50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:01.234168 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c877f35-8c96-408a-9ebc-8ff6f2a38b50 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:01.261069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1375/2756] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:34:01 2022] POST /identity/v3/users => generated 298 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:34:01.265507 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316fe3a-7f5d-4f97-902a-c46907cd25fe admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:01.281339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316fe3a-7f5d-4f97-902a-c46907cd25fe admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:01.288736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482af61d-81c1-4b90-a745-7e7b13ee4567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:01.289462 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482af61d-81c1-4b90-a745-7e7b13ee4567 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:01.289806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482af61d-81c1-4b90-a745-7e7b13ee4567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:01.290113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482af61d-81c1-4b90-a745-7e7b13ee4567 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:01.290804 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482af61d-81c1-4b90-a745-7e7b13ee4567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:01.292719 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482af61d-81c1-4b90-a745-7e7b13ee4567 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:01.306256 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1382/2757] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:34:01.311562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c877f35-8c96-408a-9ebc-8ff6f2a38b50 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:01.324267 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c877f35-8c96-408a-9ebc-8ff6f2a38b50 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:01.329789 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646c40c-772d-4b0e-a793-80008e7848ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:01.330488 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646c40c-772d-4b0e-a793-80008e7848ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:01.330771 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646c40c-772d-4b0e-a793-80008e7848ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:01.331045 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5646c40c-772d-4b0e-a793-80008e7848ed admin admin] PATH_INFO: `/v3/projects/c1f2aa38ad0248cb9661edc469252361/users/98e071c1b74b4e03b46789d7f75dc0aa/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:01.352096 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646c40c-772d-4b0e-a793-80008e7848ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f2aa38ad0248cb9661edc469252361, user_id=98e071c1b74b4e03b46789d7f75dc0aa, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:01.357706 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5646c40c-772d-4b0e-a793-80008e7848ed admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:01.385400 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1376/2758] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:34:01 2022] PUT /identity/v3/projects/c1f2aa38ad0248cb9661edc469252361/users/98e071c1b74b4e03b46789d7f75dc0aa/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:34:01.388392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482af61d-81c1-4b90-a745-7e7b13ee4567 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:01.407513 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482af61d-81c1-4b90-a745-7e7b13ee4567 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:01.509513 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9df936-b099-4fe8-ab95-30b4adb9d023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:01.510234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9df936-b099-4fe8-ab95-30b4adb9d023 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:01.510620 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9df936-b099-4fe8-ab95-30b4adb9d023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:01.510866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9df936-b099-4fe8-ab95-30b4adb9d023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:01.511627 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9df936-b099-4fe8-ab95-30b4adb9d023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:01.513405 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9df936-b099-4fe8-ab95-30b4adb9d023 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:01.547712 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1383/2759] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:34:01.549672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646c40c-772d-4b0e-a793-80008e7848ed admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:01.569201 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5646c40c-772d-4b0e-a793-80008e7848ed admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:01.577719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea144f1-3748-4842-8539-ffb299833f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:01.578863 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea144f1-3748-4842-8539-ffb299833f31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:01.579336 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea144f1-3748-4842-8539-ffb299833f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:01.579811 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea144f1-3748-4842-8539-ffb299833f31 admin admin] PATH_INFO: `/v3/projects/c1f2aa38ad0248cb9661edc469252361/users/98e071c1b74b4e03b46789d7f75dc0aa/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:01.587659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea144f1-3748-4842-8539-ffb299833f31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f2aa38ad0248cb9661edc469252361, user_id=98e071c1b74b4e03b46789d7f75dc0aa, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:01.594820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea144f1-3748-4842-8539-ffb299833f31 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:01.609141 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1377/2760] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:34:01 2022] PUT /identity/v3/projects/c1f2aa38ad0248cb9661edc469252361/users/98e071c1b74b4e03b46789d7f75dc0aa/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:34:03.083237 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3442823-55f0-4bd2-afff-30a96179d1e6 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:03.084181 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3442823-55f0-4bd2-afff-30a96179d1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:03.084181 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3442823-55f0-4bd2-afff-30a96179d1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:03.132318 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-f3442823-55f0-4bd2-afff-30a96179d1e6 None None] MFA Rules not processed for user `656a586e718543bfb095790441c28fe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:03.133961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f3442823-55f0-4bd2-afff-30a96179d1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:03.250083 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1384/2761] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:34:09.419922 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b9a51-d286-48f3-9454-6b4d70ac2960 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:09.426603 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b9a51-d286-48f3-9454-6b4d70ac2960 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:09.427706 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b9a51-d286-48f3-9454-6b4d70ac2960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:09.500245 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-4f3b9a51-d286-48f3-9454-6b4d70ac2960 None None] MFA Rules not processed for user `98e071c1b74b4e03b46789d7f75dc0aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:09.502299 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4f3b9a51-d286-48f3-9454-6b4d70ac2960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:09.740982 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:09.746953 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1378/2762] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:34:09.770858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3442823-55f0-4bd2-afff-30a96179d1e6 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:09.773727 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f3442823-55f0-4bd2-afff-30a96179d1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:10.067283 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3442823-55f0-4bd2-afff-30a96179d1e6 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:10.094590 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d617f-2a4d-4b04-a4cb-02e2e5c8ae96 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9daa1a3c7654dcfa5ccb03f467ce7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:10.095703 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d617f-2a4d-4b04-a4cb-02e2e5c8ae96 service glance] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:10.096254 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d617f-2a4d-4b04-a4cb-02e2e5c8ae96 service glance] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:10.096741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d617f-2a4d-4b04-a4cb-02e2e5c8ae96 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:10.114275 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:10.229888 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d617f-2a4d-4b04-a4cb-02e2e5c8ae96 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:10.232878 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d617f-2a4d-4b04-a4cb-02e2e5c8ae96 service glance] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:10.252424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1385/2763] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:34:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:34:10.532122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348d11d-b286-43e0-94e5-b56661d2fa0c None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:10.534002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348d11d-b286-43e0-94e5-b56661d2fa0c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:10.535052 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348d11d-b286-43e0-94e5-b56661d2fa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:10.586954 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-6348d11d-b286-43e0-94e5-b56661d2fa0c None None] MFA Rules not processed for user `61ff29258a214817b8fc15b51fe8b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:10.588319 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-6348d11d-b286-43e0-94e5-b56661d2fa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:10.670314 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1379/2764] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Aug 7 19:34:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:34:10.699921 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d617f-2a4d-4b04-a4cb-02e2e5c8ae96 service glance] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:10.702670 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d53d617f-2a4d-4b04-a4cb-02e2e5c8ae96 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:10.835819 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d617f-2a4d-4b04-a4cb-02e2e5c8ae96 service glance] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:10.846033 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41114c6-39a9-4e1f-b52b-8ba75dca72a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db51971158a946c98a8ed716ecc5d72b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:10.847263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41114c6-39a9-4e1f-b52b-8ba75dca72a3 service swift] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:10.847930 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41114c6-39a9-4e1f-b52b-8ba75dca72a3 service swift] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:10.848467 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41114c6-39a9-4e1f-b52b-8ba75dca72a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:10.871006 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41114c6-39a9-4e1f-b52b-8ba75dca72a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:10.873925 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41114c6-39a9-4e1f-b52b-8ba75dca72a3 service swift] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:10.891314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1386/2765] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Aug 7 19:34:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 19:34:20.663080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=32 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:20.883581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=55 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:21.403970 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348d11d-b286-43e0-94e5-b56661d2fa0c None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:21.406319 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-6348d11d-b286-43e0-94e5-b56661d2fa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:21.517059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348d11d-b286-43e0-94e5-b56661d2fa0c None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:21.523421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ac126-c474-416b-bba4-941368f83509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:21.524584 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ac126-c474-416b-bba4-941368f83509 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:21.525209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ac126-c474-416b-bba4-941368f83509 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:21.525810 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ac126-c474-416b-bba4-941368f83509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:21.529253 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-ed0ac126-c474-416b-bba4-941368f83509 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:21.747267 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ac126-c474-416b-bba4-941368f83509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:21.765094 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ac126-c474-416b-bba4-941368f83509 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:21.819358 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1380/2766] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:34:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:34:24.236267 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607fec8-3d35-45bf-9818-974adb003c6a None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:24.236946 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607fec8-3d35-45bf-9818-974adb003c6a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:24.237612 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607fec8-3d35-45bf-9818-974adb003c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:24.280090 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-9607fec8-3d35-45bf-9818-974adb003c6a None None] MFA Rules not processed for user `523936e9a8ca49b9805fa110a1841d64`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:24.282113 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9607fec8-3d35-45bf-9818-974adb003c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:24.304279 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1387/2767] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:34:25.289418 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ac126-c474-416b-bba4-941368f83509 service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:25.330922 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ac126-c474-416b-bba4-941368f83509 service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:25.347671 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d8067-901c-433b-9800-271b07695135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:25.348690 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d8067-901c-433b-9800-271b07695135 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:25.349179 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d8067-901c-433b-9800-271b07695135 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:25.349671 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d8067-901c-433b-9800-271b07695135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:25.352816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-584d8067-901c-433b-9800-271b07695135 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:25.476836 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d8067-901c-433b-9800-271b07695135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:25.478743 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d8067-901c-433b-9800-271b07695135 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:25.492794 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1381/2768] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:34:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:34:26.563803 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607fec8-3d35-45bf-9818-974adb003c6a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:26.566461 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9607fec8-3d35-45bf-9818-974adb003c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:26.695420 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607fec8-3d35-45bf-9818-974adb003c6a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:26.701656 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea475ca2-27b5-4456-b7ed-ea7c2b3b3d6d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '143c30b4201645279564e72ab3288adc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:26.702299 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea475ca2-27b5-4456-b7ed-ea7c2b3b3d6d service placement] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:26.702571 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea475ca2-27b5-4456-b7ed-ea7c2b3b3d6d service placement] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:26.702848 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea475ca2-27b5-4456-b7ed-ea7c2b3b3d6d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:26.704947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ea475ca2-27b5-4456-b7ed-ea7c2b3b3d6d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:26.718634 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea475ca2-27b5-4456-b7ed-ea7c2b3b3d6d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:26.720953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea475ca2-27b5-4456-b7ed-ea7c2b3b3d6d service placement] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:26.735505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1388/2769] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:34:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:34:28.048354 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613c78c-c0e8-4e16-bd4b-08066b512868 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:28.049786 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613c78c-c0e8-4e16-bd4b-08066b512868 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:28.050828 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613c78c-c0e8-4e16-bd4b-08066b512868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:28.074115 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:28.082388 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-5613c78c-c0e8-4e16-bd4b-08066b512868 None None] MFA Rules not processed for user `61ff29258a214817b8fc15b51fe8b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:28.085339 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-5613c78c-c0e8-4e16-bd4b-08066b512868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:28.108430 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1382/2770] 10.222.0.17 () {58 vars in 1009 bytes} [Sun Aug 7 19:34:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:34:28.116911 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea475ca2-27b5-4456-b7ed-ea7c2b3b3d6d service placement] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:28.129203 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea475ca2-27b5-4456-b7ed-ea7c2b3b3d6d service placement] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:28.136690 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a2655-d1c0-42e1-ba76-1d588e3899c2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db51971158a946c98a8ed716ecc5d72b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:28.137338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a2655-d1c0-42e1-ba76-1d588e3899c2 service swift] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:28.137615 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a2655-d1c0-42e1-ba76-1d588e3899c2 service swift] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:28.137880 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a2655-d1c0-42e1-ba76-1d588e3899c2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:28.169994 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a2655-d1c0-42e1-ba76-1d588e3899c2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:28.174234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a2655-d1c0-42e1-ba76-1d588e3899c2 service swift] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:28.194752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1389/2771] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Aug 7 19:34:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 07 19:34:28.671083 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613c78c-c0e8-4e16-bd4b-08066b512868 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:28.673397 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-5613c78c-c0e8-4e16-bd4b-08066b512868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:28.706245 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613c78c-c0e8-4e16-bd4b-08066b512868 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:28.717125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f533b7-e621-40e9-8597-d1b853f624e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:28.718186 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f533b7-e621-40e9-8597-d1b853f624e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:28.718512 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f533b7-e621-40e9-8597-d1b853f624e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:28.718773 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f533b7-e621-40e9-8597-d1b853f624e2 admin admin] PATH_INFO: `/v3/users/bb092d77de334d3ab631a42979460c59` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:28.720309 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f533b7-e621-40e9-8597-d1b853f624e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb092d77de334d3ab631a42979460c59)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:28.722384 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f533b7-e621-40e9-8597-d1b853f624e2 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:28.845486 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-74f533b7-e621-40e9-8597-d1b853f624e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb092d77de334d3ab631a42979460c59'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:28.881760 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-74f533b7-e621-40e9-8597-d1b853f624e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb092d77de334d3ab631a42979460c59'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:28.916955 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-74f533b7-e621-40e9-8597-d1b853f624e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb092d77de334d3ab631a42979460c59'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:28.924811 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-74f533b7-e621-40e9-8597-d1b853f624e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb092d77de334d3ab631a42979460c59'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:28.937565 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-74f533b7-e621-40e9-8597-d1b853f624e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb092d77de334d3ab631a42979460c59'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:28.953675 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1383/2772] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:34:28 2022] DELETE /identity/v3/users/bb092d77de334d3ab631a42979460c59 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:34:29.444200 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a2655-d1c0-42e1-ba76-1d588e3899c2 service swift] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:29.461754 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a2655-d1c0-42e1-ba76-1d588e3899c2 service swift] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:29.605483 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad873a7a-b977-4d5d-adb4-13ba757d9f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:29.606190 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad873a7a-b977-4d5d-adb4-13ba757d9f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:29.607356 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad873a7a-b977-4d5d-adb4-13ba757d9f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:29.607951 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad873a7a-b977-4d5d-adb4-13ba757d9f41 admin admin] PATH_INFO: `/v3/projects/d12435b63a424deeb1cca3ba959245a4` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:29.609760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad873a7a-b977-4d5d-adb4-13ba757d9f41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d12435b63a424deeb1cca3ba959245a4)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:29.613271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad873a7a-b977-4d5d-adb4-13ba757d9f41 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:29.695543 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ad873a7a-b977-4d5d-adb4-13ba757d9f41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd12435b63a424deeb1cca3ba959245a4', 'request_id': 'req-ad873a7a-b977-4d5d-adb4-13ba757d9f41'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:29.698893 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:29.706329 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ad873a7a-b977-4d5d-adb4-13ba757d9f41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd12435b63a424deeb1cca3ba959245a4', 'request_id': 'req-ad873a7a-b977-4d5d-adb4-13ba757d9f41'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:29.718714 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ad873a7a-b977-4d5d-adb4-13ba757d9f41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd12435b63a424deeb1cca3ba959245a4', 'request_id': 'req-ad873a7a-b977-4d5d-adb4-13ba757d9f41'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:29.777038 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ad873a7a-b977-4d5d-adb4-13ba757d9f41 admin admin] The token cache is being invalidate because project d12435b63a424deeb1cca3ba959245a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:34:29.778726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ad873a7a-b977-4d5d-adb4-13ba757d9f41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:29.783721 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-ad873a7a-b977-4d5d-adb4-13ba757d9f41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:29.797252 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1390/2773] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:34:29 2022] DELETE /identity/v3/projects/d12435b63a424deeb1cca3ba959245a4 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:34:29.807046 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c66c1-97cb-403c-ac86-90ddb5da389e None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:29.813911 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c66c1-97cb-403c-ac86-90ddb5da389e None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:29.816208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c66c1-97cb-403c-ac86-90ddb5da389e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:29.990323 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-df3c66c1-97cb-403c-ac86-90ddb5da389e None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:29.994221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-df3c66c1-97cb-403c-ac86-90ddb5da389e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:30.154331 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1384/2774] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:34:30.156908 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b2caa-6331-471f-a399-7a7552683e2a None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:30.157273 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b2caa-6331-471f-a399-7a7552683e2a None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:30.157538 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b2caa-6331-471f-a399-7a7552683e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:30.202274 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-b71b2caa-6331-471f-a399-7a7552683e2a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:30.203900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-b71b2caa-6331-471f-a399-7a7552683e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:30.229606 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1391/2775] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:34:30.231551 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c66c1-97cb-403c-ac86-90ddb5da389e None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:30.245385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c66c1-97cb-403c-ac86-90ddb5da389e None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:30.253147 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1563f8-0c4f-4913-8044-86922a84de42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:30.254555 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1563f8-0c4f-4913-8044-86922a84de42 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:30.255243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1563f8-0c4f-4913-8044-86922a84de42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:30.255899 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1563f8-0c4f-4913-8044-86922a84de42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:30.257256 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1563f8-0c4f-4913-8044-86922a84de42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:34:30.257890 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1563f8-0c4f-4913-8044-86922a84de42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:30.261295 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1563f8-0c4f-4913-8044-86922a84de42 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:30.293266 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1385/2776] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:34:30.295860 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b2caa-6331-471f-a399-7a7552683e2a None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:30.309715 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b2caa-6331-471f-a399-7a7552683e2a None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:30.317335 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af15af-a0ab-480f-9fe3-b73b28e9020d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:30.318606 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af15af-a0ab-480f-9fe3-b73b28e9020d admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:30.319182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af15af-a0ab-480f-9fe3-b73b28e9020d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:30.319718 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af15af-a0ab-480f-9fe3-b73b28e9020d admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:30.320621 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af15af-a0ab-480f-9fe3-b73b28e9020d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:30.324833 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af15af-a0ab-480f-9fe3-b73b28e9020d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:30.379893 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1392/2777] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:34:30 2022] POST /identity/v3/projects => generated 365 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:34:30.384709 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1563f8-0c4f-4913-8044-86922a84de42 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:30.406330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1563f8-0c4f-4913-8044-86922a84de42 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:30.540646 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:30.601364 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb361d6-c22a-4a99-87eb-d41ce5d1ce51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:30.602554 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb361d6-c22a-4a99-87eb-d41ce5d1ce51 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:30.603159 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb361d6-c22a-4a99-87eb-d41ce5d1ce51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:30.603705 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb361d6-c22a-4a99-87eb-d41ce5d1ce51 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:30.605037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb361d6-c22a-4a99-87eb-d41ce5d1ce51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:30.609083 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb361d6-c22a-4a99-87eb-d41ce5d1ce51 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:30.663704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1386/2778] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:34:30 2022] POST /identity/v3/users => generated 283 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:34:30.663979 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af15af-a0ab-480f-9fe3-b73b28e9020d admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:30.684648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af15af-a0ab-480f-9fe3-b73b28e9020d admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:30.703164 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38851ae8-217c-48e8-bdae-3277cf8bff88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:30.704133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38851ae8-217c-48e8-bdae-3277cf8bff88 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:30.704339 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38851ae8-217c-48e8-bdae-3277cf8bff88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:30.704769 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38851ae8-217c-48e8-bdae-3277cf8bff88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:30.705616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38851ae8-217c-48e8-bdae-3277cf8bff88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:30.708387 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38851ae8-217c-48e8-bdae-3277cf8bff88 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:30.740444 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1393/2779] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:34:30.743636 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb361d6-c22a-4a99-87eb-d41ce5d1ce51 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:30.771345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb361d6-c22a-4a99-87eb-d41ce5d1ce51 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:30.804922 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44856e2b-11b1-47b6-943f-1f7d43912866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:30.805784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44856e2b-11b1-47b6-943f-1f7d43912866 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:30.806186 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44856e2b-11b1-47b6-943f-1f7d43912866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:30.806595 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44856e2b-11b1-47b6-943f-1f7d43912866 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:30.807949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44856e2b-11b1-47b6-943f-1f7d43912866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:30.809924 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44856e2b-11b1-47b6-943f-1f7d43912866 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:30.836464 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1387/2780] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:34:30.841512 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38851ae8-217c-48e8-bdae-3277cf8bff88 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:30.876102 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38851ae8-217c-48e8-bdae-3277cf8bff88 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:30.885933 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cbfa2d-19c8-4f74-8ea2-2b9b45b8f3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:30.886980 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbfa2d-19c8-4f74-8ea2-2b9b45b8f3b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:30.887379 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbfa2d-19c8-4f74-8ea2-2b9b45b8f3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:30.895046 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cbfa2d-19c8-4f74-8ea2-2b9b45b8f3b7 admin admin] PATH_INFO: `/v3/projects/65d3e36589874e8ebfaafb149614f7de/users/a9b1306526fd4c63a79c2983c9e0785b/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:30.943483 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cbfa2d-19c8-4f74-8ea2-2b9b45b8f3b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65d3e36589874e8ebfaafb149614f7de, user_id=a9b1306526fd4c63a79c2983c9e0785b, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:30.947959 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cbfa2d-19c8-4f74-8ea2-2b9b45b8f3b7 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:30.980022 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1394/2781] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:34:30 2022] PUT /identity/v3/projects/65d3e36589874e8ebfaafb149614f7de/users/a9b1306526fd4c63a79c2983c9e0785b/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:34:30.991557 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72200b1b-555b-4e21-8a93-cc2fda87bff3 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:30.992115 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72200b1b-555b-4e21-8a93-cc2fda87bff3 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:30.992521 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72200b1b-555b-4e21-8a93-cc2fda87bff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:31.078793 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-72200b1b-555b-4e21-8a93-cc2fda87bff3 None None] MFA Rules not processed for user `a9b1306526fd4c63a79c2983c9e0785b`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:31.083296 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-72200b1b-555b-4e21-8a93-cc2fda87bff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:31.247601 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1388/2782] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:34:31.252873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cbfa2d-19c8-4f74-8ea2-2b9b45b8f3b7 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:31.444389 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cbfa2d-19c8-4f74-8ea2-2b9b45b8f3b7 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:31.464898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6a8bd-4d11-4c69-afd6-ebe62e745628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:31.465579 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6a8bd-4d11-4c69-afd6-ebe62e745628 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:31.465888 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6a8bd-4d11-4c69-afd6-ebe62e745628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:31.466182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6a8bd-4d11-4c69-afd6-ebe62e745628 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:31.467201 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6a8bd-4d11-4c69-afd6-ebe62e745628 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:31.481975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f6a8bd-4d11-4c69-afd6-ebe62e745628 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:31.561705 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1395/2783] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:34:31 2022] POST /identity/v3/projects => generated 365 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:34:31.564478 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72200b1b-555b-4e21-8a93-cc2fda87bff3 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:31.583452 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72200b1b-555b-4e21-8a93-cc2fda87bff3 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:31.790226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366839b-4431-422a-aea9-34e80d248bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:31.790806 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366839b-4431-422a-aea9-34e80d248bbc admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:31.791140 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366839b-4431-422a-aea9-34e80d248bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:31.791428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366839b-4431-422a-aea9-34e80d248bbc admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:31.792292 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366839b-4431-422a-aea9-34e80d248bbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:31.795689 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366839b-4431-422a-aea9-34e80d248bbc admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:31.850507 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1389/2784] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:34:31 2022] POST /identity/v3/users => generated 289 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:34:31.852736 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6a8bd-4d11-4c69-afd6-ebe62e745628 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:31.882861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f6a8bd-4d11-4c69-afd6-ebe62e745628 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:31.899225 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e625d-39a8-4777-ba10-ff611c7076e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:31.899719 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e625d-39a8-4777-ba10-ff611c7076e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:31.899972 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e625d-39a8-4777-ba10-ff611c7076e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:31.900285 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e625d-39a8-4777-ba10-ff611c7076e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:31.900988 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e625d-39a8-4777-ba10-ff611c7076e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:31.902691 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e625d-39a8-4777-ba10-ff611c7076e4 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:31.931325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1396/2785] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:34:31.936191 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366839b-4431-422a-aea9-34e80d248bbc admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:31.949949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366839b-4431-422a-aea9-34e80d248bbc admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:31.966842 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a82eae-493c-410d-acea-367be697d3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:31.966842 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a82eae-493c-410d-acea-367be697d3a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:31.967207 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a82eae-493c-410d-acea-367be697d3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:31.970302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a82eae-493c-410d-acea-367be697d3a4 admin admin] PATH_INFO: `/v3/projects/5768025ee1b84050ac386044fd604d29/users/e664321471e7409590334790010193e9/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:32.012560 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a82eae-493c-410d-acea-367be697d3a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5768025ee1b84050ac386044fd604d29, user_id=e664321471e7409590334790010193e9, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:32.015430 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a82eae-493c-410d-acea-367be697d3a4 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:32.043371 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1390/2786] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:34:31 2022] PUT /identity/v3/projects/5768025ee1b84050ac386044fd604d29/users/e664321471e7409590334790010193e9/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:34:32.049131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e625d-39a8-4777-ba10-ff611c7076e4 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:32.062105 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e625d-39a8-4777-ba10-ff611c7076e4 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:32.280157 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:32.325677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7d2017-818e-469a-b688-89c1ec5f5ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:32.327689 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d2017-818e-469a-b688-89c1ec5f5ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:32.328469 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d2017-818e-469a-b688-89c1ec5f5ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:32.329182 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7d2017-818e-469a-b688-89c1ec5f5ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:32.333140 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d2017-818e-469a-b688-89c1ec5f5ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:32.341518 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7d2017-818e-469a-b688-89c1ec5f5ba5 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:32.369951 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1397/2787] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:34:32 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:34:32.373343 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a82eae-493c-410d-acea-367be697d3a4 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:32.381260 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:32.396051 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a82eae-493c-410d-acea-367be697d3a4 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:32.410302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b3d21-f16e-4afe-aff1-484223405980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:32.411617 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b3d21-f16e-4afe-aff1-484223405980 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:32.413639 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b3d21-f16e-4afe-aff1-484223405980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:32.415211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b3d21-f16e-4afe-aff1-484223405980 admin admin] PATH_INFO: `/v3/projects/5768025ee1b84050ac386044fd604d29/users/e664321471e7409590334790010193e9/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:32.422306 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b3d21-f16e-4afe-aff1-484223405980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5768025ee1b84050ac386044fd604d29, user_id=e664321471e7409590334790010193e9, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:32.429241 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b3d21-f16e-4afe-aff1-484223405980 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:32.478489 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1391/2788] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:34:32 2022] PUT /identity/v3/projects/5768025ee1b84050ac386044fd604d29/users/e664321471e7409590334790010193e9/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:34:32.480623 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169cafe-f6d2-45ea-82a3-25344ab9d777 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:32.481218 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169cafe-f6d2-45ea-82a3-25344ab9d777 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:32.481702 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169cafe-f6d2-45ea-82a3-25344ab9d777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:32.608881 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-f169cafe-f6d2-45ea-82a3-25344ab9d777 None None] MFA Rules not processed for user `e664321471e7409590334790010193e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:32.614496 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f169cafe-f6d2-45ea-82a3-25344ab9d777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:32.789729 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1398/2789] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:34:32.804069 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b3d21-f16e-4afe-aff1-484223405980 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:32.807375 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-ff4b3d21-f16e-4afe-aff1-484223405980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:32.971312 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b3d21-f16e-4afe-aff1-484223405980 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:32.976036 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634de376-bffc-43d3-985c-72844fb57859 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:32.976828 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634de376-bffc-43d3-985c-72844fb57859 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:32.977348 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634de376-bffc-43d3-985c-72844fb57859 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:32.977726 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634de376-bffc-43d3-985c-72844fb57859 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:32.989509 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634de376-bffc-43d3-985c-72844fb57859 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:32.991295 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634de376-bffc-43d3-985c-72844fb57859 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:33.014480 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1392/2790] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:34:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:34:33.238426 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169cafe-f6d2-45ea-82a3-25344ab9d777 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:33.256170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169cafe-f6d2-45ea-82a3-25344ab9d777 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:33.263937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2aa19-35f6-45b7-8567-546a08d18f28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:33.267395 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2aa19-35f6-45b7-8567-546a08d18f28 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:33.267811 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2aa19-35f6-45b7-8567-546a08d18f28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:33.268309 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2aa19-35f6-45b7-8567-546a08d18f28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:33.429175 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec2aa19-35f6-45b7-8567-546a08d18f28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:33.430894 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec2aa19-35f6-45b7-8567-546a08d18f28 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:33.456126 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1399/2791] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:34:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:34:36.925319 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634de376-bffc-43d3-985c-72844fb57859 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:37.050390 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634de376-bffc-43d3-985c-72844fb57859 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:37.058853 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f7cd6-74d8-42be-80f7-c68b6dff9fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:37.059570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f7cd6-74d8-42be-80f7-c68b6dff9fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:37.059880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f7cd6-74d8-42be-80f7-c68b6dff9fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:37.060310 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f7cd6-74d8-42be-80f7-c68b6dff9fa3 admin admin] PATH_INFO: `/v3/users/a9b1306526fd4c63a79c2983c9e0785b` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:37.061715 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f7cd6-74d8-42be-80f7-c68b6dff9fa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9b1306526fd4c63a79c2983c9e0785b)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:37.063805 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889f7cd6-74d8-42be-80f7-c68b6dff9fa3 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:37.121977 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-889f7cd6-74d8-42be-80f7-c68b6dff9fa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9b1306526fd4c63a79c2983c9e0785b'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:37.129798 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-889f7cd6-74d8-42be-80f7-c68b6dff9fa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9b1306526fd4c63a79c2983c9e0785b'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:37.142492 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-889f7cd6-74d8-42be-80f7-c68b6dff9fa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9b1306526fd4c63a79c2983c9e0785b'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:37.143136 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-889f7cd6-74d8-42be-80f7-c68b6dff9fa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9b1306526fd4c63a79c2983c9e0785b'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:37.145117 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:37.155381 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-889f7cd6-74d8-42be-80f7-c68b6dff9fa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9b1306526fd4c63a79c2983c9e0785b'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:37.165594 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1393/2792] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:34:36 2022] DELETE /identity/v3/users/a9b1306526fd4c63a79c2983c9e0785b => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:34:37.170886 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b041fdd-228b-4019-bf98-a0fab90ffa8c None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:37.171553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b041fdd-228b-4019-bf98-a0fab90ffa8c None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:37.171553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b041fdd-228b-4019-bf98-a0fab90ffa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:37.192037 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-8b041fdd-228b-4019-bf98-a0fab90ffa8c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:37.193297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8b041fdd-228b-4019-bf98-a0fab90ffa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:37.260250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1400/2793] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:34:37.270758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f7cd6-74d8-42be-80f7-c68b6dff9fa3 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:37.272370 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-889f7cd6-74d8-42be-80f7-c68b6dff9fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:37.371934 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889f7cd6-74d8-42be-80f7-c68b6dff9fa3 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:37.378091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78044e67-b812-4656-aba9-fc84e3540819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:37.379077 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78044e67-b812-4656-aba9-fc84e3540819 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:37.379570 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78044e67-b812-4656-aba9-fc84e3540819 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:37.380065 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78044e67-b812-4656-aba9-fc84e3540819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:37.396631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78044e67-b812-4656-aba9-fc84e3540819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:37.399388 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78044e67-b812-4656-aba9-fc84e3540819 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:37.415081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1394/2794] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:34:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:34:37.929383 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b041fdd-228b-4019-bf98-a0fab90ffa8c None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:37.931687 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-8b041fdd-228b-4019-bf98-a0fab90ffa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:37.955388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b041fdd-228b-4019-bf98-a0fab90ffa8c None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:37.964371 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26819934-89d7-437d-a07b-486e65824385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:37.968999 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26819934-89d7-437d-a07b-486e65824385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:37.968999 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26819934-89d7-437d-a07b-486e65824385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:37.968999 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26819934-89d7-437d-a07b-486e65824385 admin admin] PATH_INFO: `/v3/projects/65d3e36589874e8ebfaafb149614f7de` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:37.969845 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26819934-89d7-437d-a07b-486e65824385 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65d3e36589874e8ebfaafb149614f7de)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:37.973407 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26819934-89d7-437d-a07b-486e65824385 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:37.988319 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=100 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:38.090630 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-26819934-89d7-437d-a07b-486e65824385 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65d3e36589874e8ebfaafb149614f7de', 'request_id': 'req-26819934-89d7-437d-a07b-486e65824385'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:38.098681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-26819934-89d7-437d-a07b-486e65824385 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65d3e36589874e8ebfaafb149614f7de', 'request_id': 'req-26819934-89d7-437d-a07b-486e65824385'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:38.109641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-26819934-89d7-437d-a07b-486e65824385 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65d3e36589874e8ebfaafb149614f7de', 'request_id': 'req-26819934-89d7-437d-a07b-486e65824385'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:38.123857 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-26819934-89d7-437d-a07b-486e65824385 admin admin] The token cache is being invalidate because project 65d3e36589874e8ebfaafb149614f7de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:34:38.124210 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-26819934-89d7-437d-a07b-486e65824385 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:38.124923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-26819934-89d7-437d-a07b-486e65824385 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:38.126510 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1401/2795] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:34:37 2022] DELETE /identity/v3/projects/65d3e36589874e8ebfaafb149614f7de => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:34:38.131017 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78044e67-b812-4656-aba9-fc84e3540819 service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:38.142727 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78044e67-b812-4656-aba9-fc84e3540819 service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:38.219698 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208585c9-910c-446f-b560-9e669789833b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:38.220491 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208585c9-910c-446f-b560-9e669789833b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:38.220814 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208585c9-910c-446f-b560-9e669789833b admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:38.221127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208585c9-910c-446f-b560-9e669789833b admin admin] PATH_INFO: `/v3/users/e664321471e7409590334790010193e9` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:38.222610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208585c9-910c-446f-b560-9e669789833b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e664321471e7409590334790010193e9)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:38.224898 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208585c9-910c-446f-b560-9e669789833b admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:38.292824 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-208585c9-910c-446f-b560-9e669789833b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e664321471e7409590334790010193e9'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:38.303458 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-208585c9-910c-446f-b560-9e669789833b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e664321471e7409590334790010193e9'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:38.318447 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-208585c9-910c-446f-b560-9e669789833b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e664321471e7409590334790010193e9'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:38.319956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-208585c9-910c-446f-b560-9e669789833b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e664321471e7409590334790010193e9'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:38.329339 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-208585c9-910c-446f-b560-9e669789833b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e664321471e7409590334790010193e9'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:38.336653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1395/2796] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:34:38 2022] DELETE /identity/v3/users/e664321471e7409590334790010193e9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:34:38.684145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26819934-89d7-437d-a07b-486e65824385 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:38.685896 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-26819934-89d7-437d-a07b-486e65824385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:38.701427 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26819934-89d7-437d-a07b-486e65824385 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:38.771305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38992215-87ed-42a5-9c63-531849bf3e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:38.771972 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38992215-87ed-42a5-9c63-531849bf3e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:38.772551 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38992215-87ed-42a5-9c63-531849bf3e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:38.772826 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38992215-87ed-42a5-9c63-531849bf3e22 admin admin] PATH_INFO: `/v3/projects/5768025ee1b84050ac386044fd604d29` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:38.774112 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38992215-87ed-42a5-9c63-531849bf3e22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5768025ee1b84050ac386044fd604d29)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:38.776144 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38992215-87ed-42a5-9c63-531849bf3e22 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:38.830846 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-38992215-87ed-42a5-9c63-531849bf3e22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5768025ee1b84050ac386044fd604d29', 'request_id': 'req-38992215-87ed-42a5-9c63-531849bf3e22'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:38.838782 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-38992215-87ed-42a5-9c63-531849bf3e22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5768025ee1b84050ac386044fd604d29', 'request_id': 'req-38992215-87ed-42a5-9c63-531849bf3e22'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:38.849216 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-38992215-87ed-42a5-9c63-531849bf3e22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5768025ee1b84050ac386044fd604d29', 'request_id': 'req-38992215-87ed-42a5-9c63-531849bf3e22'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:38.863593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-38992215-87ed-42a5-9c63-531849bf3e22 admin admin] The token cache is being invalidate because project 5768025ee1b84050ac386044fd604d29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:34:38.863895 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-38992215-87ed-42a5-9c63-531849bf3e22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:38.864756 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-38992215-87ed-42a5-9c63-531849bf3e22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:38.866616 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1402/2797] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:34:38 2022] DELETE /identity/v3/projects/5768025ee1b84050ac386044fd604d29 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:34:38.873037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882f98d-e4b2-42d1-88c3-5a173b99265a None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:38.873349 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882f98d-e4b2-42d1-88c3-5a173b99265a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:38.873686 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882f98d-e4b2-42d1-88c3-5a173b99265a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:38.882846 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:38.903865 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-5882f98d-e4b2-42d1-88c3-5a173b99265a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:38.907789 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-5882f98d-e4b2-42d1-88c3-5a173b99265a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:39.012097 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1396/2798] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:34:39.025011 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cb23f-ca51-4d69-bc59-b08694bc942e None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:39.025449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cb23f-ca51-4d69-bc59-b08694bc942e None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:39.025815 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720cb23f-ca51-4d69-bc59-b08694bc942e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:39.050244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-720cb23f-ca51-4d69-bc59-b08694bc942e None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:39.051537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-720cb23f-ca51-4d69-bc59-b08694bc942e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:39.061111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1403/2799] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:34:39.066640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882f98d-e4b2-42d1-88c3-5a173b99265a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:39.085175 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882f98d-e4b2-42d1-88c3-5a173b99265a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:39.093206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fbb3d-8862-41b0-86da-defc66fed86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:39.094273 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fbb3d-8862-41b0-86da-defc66fed86f admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:39.094753 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fbb3d-8862-41b0-86da-defc66fed86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:39.095250 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fbb3d-8862-41b0-86da-defc66fed86f admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:39.096514 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fbb3d-8862-41b0-86da-defc66fed86f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:34:39.097322 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fbb3d-8862-41b0-86da-defc66fed86f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:39.100489 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3fbb3d-8862-41b0-86da-defc66fed86f admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:39.119642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1397/2800] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:34:39.126080 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720cb23f-ca51-4d69-bc59-b08694bc942e None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:39.138358 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720cb23f-ca51-4d69-bc59-b08694bc942e None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:39.144065 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b7e64-4ae8-4bd9-91ff-b789f5df5ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:39.144849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b7e64-4ae8-4bd9-91ff-b789f5df5ded admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:39.145228 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b7e64-4ae8-4bd9-91ff-b789f5df5ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:39.145631 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b7e64-4ae8-4bd9-91ff-b789f5df5ded admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:39.146589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b7e64-4ae8-4bd9-91ff-b789f5df5ded admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:39.148840 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b7e64-4ae8-4bd9-91ff-b789f5df5ded admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:39.179885 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:39.201251 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1404/2801] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:34:39 2022] POST /identity/v3/projects => generated 401 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:34:39.205450 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fbb3d-8862-41b0-86da-defc66fed86f admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:39.217868 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3fbb3d-8862-41b0-86da-defc66fed86f admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:39.307463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291dd95-4d95-459e-b82f-3260a2ac7cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:39.308157 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291dd95-4d95-459e-b82f-3260a2ac7cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:39.308428 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291dd95-4d95-459e-b82f-3260a2ac7cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:39.308847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291dd95-4d95-459e-b82f-3260a2ac7cc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:39.309555 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291dd95-4d95-459e-b82f-3260a2ac7cc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:39.311623 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291dd95-4d95-459e-b82f-3260a2ac7cc7 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:39.338107 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1398/2802] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:34:39 2022] POST /identity/v3/users => generated 301 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:34:39.343258 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b7e64-4ae8-4bd9-91ff-b789f5df5ded admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:39.356177 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b7e64-4ae8-4bd9-91ff-b789f5df5ded admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:39.362549 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2bfc3-53c2-48fb-b6f7-d9b5119c8d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:39.363302 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2bfc3-53c2-48fb-b6f7-d9b5119c8d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:39.363628 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2bfc3-53c2-48fb-b6f7-d9b5119c8d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:39.363945 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2bfc3-53c2-48fb-b6f7-d9b5119c8d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:39.364767 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2bfc3-53c2-48fb-b6f7-d9b5119c8d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:39.366593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b2bfc3-53c2-48fb-b6f7-d9b5119c8d1f admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:39.378816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1405/2803] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:34:39.385138 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291dd95-4d95-459e-b82f-3260a2ac7cc7 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:39.401081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291dd95-4d95-459e-b82f-3260a2ac7cc7 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:39.407223 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d245c8-2036-44d4-bf13-993b3f2b414d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:39.407850 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d245c8-2036-44d4-bf13-993b3f2b414d admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:39.408171 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d245c8-2036-44d4-bf13-993b3f2b414d admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:39.408438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d245c8-2036-44d4-bf13-993b3f2b414d admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:39.409177 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d245c8-2036-44d4-bf13-993b3f2b414d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:39.410836 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d245c8-2036-44d4-bf13-993b3f2b414d admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:39.424359 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1399/2804] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:34:39.428832 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2bfc3-53c2-48fb-b6f7-d9b5119c8d1f admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:39.440975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b2bfc3-53c2-48fb-b6f7-d9b5119c8d1f admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:39.446110 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b70702-8377-4aa5-9ceb-8a4821aee7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:39.446778 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b70702-8377-4aa5-9ceb-8a4821aee7b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:39.447054 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b70702-8377-4aa5-9ceb-8a4821aee7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:39.447642 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b70702-8377-4aa5-9ceb-8a4821aee7b7 admin admin] PATH_INFO: `/v3/projects/e961955456a84871b2a100ab368872f8/users/45d9a8c4c0ea4b3b8aa38351ff55edd4/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:39.465244 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b70702-8377-4aa5-9ceb-8a4821aee7b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e961955456a84871b2a100ab368872f8, user_id=45d9a8c4c0ea4b3b8aa38351ff55edd4, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:39.468809 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b70702-8377-4aa5-9ceb-8a4821aee7b7 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:39.485907 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1406/2805] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:34:39 2022] PUT /identity/v3/projects/e961955456a84871b2a100ab368872f8/users/45d9a8c4c0ea4b3b8aa38351ff55edd4/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:34:39.498543 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22cef3-7f27-4fa4-97ab-8f79a9dd6cac None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:39.498841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22cef3-7f27-4fa4-97ab-8f79a9dd6cac None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:39.499115 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22cef3-7f27-4fa4-97ab-8f79a9dd6cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:39.532505 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-dd22cef3-7f27-4fa4-97ab-8f79a9dd6cac None None] MFA Rules not processed for user `45d9a8c4c0ea4b3b8aa38351ff55edd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:39.533642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-dd22cef3-7f27-4fa4-97ab-8f79a9dd6cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:39.603385 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:39.612464 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1400/2806] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:39 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:34:39.617406 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b70702-8377-4aa5-9ceb-8a4821aee7b7 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:39.707630 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b70702-8377-4aa5-9ceb-8a4821aee7b7 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:39.713684 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e11bce-6969-4e4e-a1a2-a1018c43bb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:39.714343 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e11bce-6969-4e4e-a1a2-a1018c43bb89 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:39.714672 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e11bce-6969-4e4e-a1a2-a1018c43bb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:39.714958 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e11bce-6969-4e4e-a1a2-a1018c43bb89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:39.715625 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e11bce-6969-4e4e-a1a2-a1018c43bb89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:39.717772 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e11bce-6969-4e4e-a1a2-a1018c43bb89 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:39.749866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1407/2807] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:34:39 2022] POST /identity/v3/projects => generated 401 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:34:39.756115 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22cef3-7f27-4fa4-97ab-8f79a9dd6cac None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:39.775864 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22cef3-7f27-4fa4-97ab-8f79a9dd6cac None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:39.850617 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c15d7d-ea79-4e14-9463-683f5ff26e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:39.851215 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c15d7d-ea79-4e14-9463-683f5ff26e35 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:39.851456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c15d7d-ea79-4e14-9463-683f5ff26e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:39.851778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c15d7d-ea79-4e14-9463-683f5ff26e35 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:39.852476 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c15d7d-ea79-4e14-9463-683f5ff26e35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:39.854568 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c15d7d-ea79-4e14-9463-683f5ff26e35 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:39.880788 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1401/2808] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:34:39 2022] POST /identity/v3/users => generated 307 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:34:39.885326 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e11bce-6969-4e4e-a1a2-a1018c43bb89 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:39.897584 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e11bce-6969-4e4e-a1a2-a1018c43bb89 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:39.903466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b2087-47e9-4d70-b9aa-57108954187c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:39.904250 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b2087-47e9-4d70-b9aa-57108954187c admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:39.904575 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b2087-47e9-4d70-b9aa-57108954187c admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:39.904861 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b2087-47e9-4d70-b9aa-57108954187c admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:39.905532 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b2087-47e9-4d70-b9aa-57108954187c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:39.907263 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b2087-47e9-4d70-b9aa-57108954187c admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:39.919278 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1408/2809] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:34:39.924910 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c15d7d-ea79-4e14-9463-683f5ff26e35 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:39.936005 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c15d7d-ea79-4e14-9463-683f5ff26e35 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:39.941188 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b84081-adc8-420c-8a80-dca7f3b9256d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:39.941886 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b84081-adc8-420c-8a80-dca7f3b9256d admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:39.942163 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b84081-adc8-420c-8a80-dca7f3b9256d admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:39.942433 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b84081-adc8-420c-8a80-dca7f3b9256d admin admin] PATH_INFO: `/v3/projects/b5d07db194d448768d84efab1c6bebcf/users/145fa08085944dc8bd9bcfe93ff1a0c2/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:39.960682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b84081-adc8-420c-8a80-dca7f3b9256d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d07db194d448768d84efab1c6bebcf, user_id=145fa08085944dc8bd9bcfe93ff1a0c2, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:39.963404 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b84081-adc8-420c-8a80-dca7f3b9256d admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:39.980669 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1402/2810] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:34:39 2022] PUT /identity/v3/projects/b5d07db194d448768d84efab1c6bebcf/users/145fa08085944dc8bd9bcfe93ff1a0c2/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:34:39.985033 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b2087-47e9-4d70-b9aa-57108954187c admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:39.997015 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b2087-47e9-4d70-b9aa-57108954187c admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:40.069503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40bc664-1285-4372-b70c-1ad2d6779c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:40.070366 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40bc664-1285-4372-b70c-1ad2d6779c3e admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:40.070757 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40bc664-1285-4372-b70c-1ad2d6779c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:40.071146 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40bc664-1285-4372-b70c-1ad2d6779c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:40.072074 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40bc664-1285-4372-b70c-1ad2d6779c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:40.076748 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40bc664-1285-4372-b70c-1ad2d6779c3e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:40.089815 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1409/2811] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:34:39 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:34:40.096114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b84081-adc8-420c-8a80-dca7f3b9256d admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:40.111369 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b84081-adc8-420c-8a80-dca7f3b9256d admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:40.118041 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7daab20-e6cf-4eff-b096-8fac4493b779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:40.118828 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daab20-e6cf-4eff-b096-8fac4493b779 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:40.119114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daab20-e6cf-4eff-b096-8fac4493b779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:40.119395 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daab20-e6cf-4eff-b096-8fac4493b779 admin admin] PATH_INFO: `/v3/projects/b5d07db194d448768d84efab1c6bebcf/users/145fa08085944dc8bd9bcfe93ff1a0c2/roles/4da42f08d107448fb877b42b61bc74ea` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:40.121540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7daab20-e6cf-4eff-b096-8fac4493b779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d07db194d448768d84efab1c6bebcf, user_id=145fa08085944dc8bd9bcfe93ff1a0c2, role_id=4da42f08d107448fb877b42b61bc74ea)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:40.124296 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7daab20-e6cf-4eff-b096-8fac4493b779 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:40.137484 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1403/2812] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:34:40 2022] PUT /identity/v3/projects/b5d07db194d448768d84efab1c6bebcf/users/145fa08085944dc8bd9bcfe93ff1a0c2/roles/4da42f08d107448fb877b42b61bc74ea => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:34:40.155406 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ce162-6ae0-498b-98b3-15143265b3b0 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:40.155743 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ce162-6ae0-498b-98b3-15143265b3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:40.156178 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ce162-6ae0-498b-98b3-15143265b3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:40.193422 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-930ce162-6ae0-498b-98b3-15143265b3b0 None None] MFA Rules not processed for user `145fa08085944dc8bd9bcfe93ff1a0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:40.194898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-930ce162-6ae0-498b-98b3-15143265b3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:40.217990 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:40.288480 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1410/2813] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:40 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:34:40.302689 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7daab20-e6cf-4eff-b096-8fac4493b779 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:40.305497 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b7daab20-e6cf-4eff-b096-8fac4493b779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:40.402736 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7daab20-e6cf-4eff-b096-8fac4493b779 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:40.406894 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bac09-9c7b-447b-ab62-ebfcb90f06ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:40.407542 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bac09-9c7b-447b-ab62-ebfcb90f06ec service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:40.407823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bac09-9c7b-447b-ab62-ebfcb90f06ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:40.408298 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bac09-9c7b-447b-ab62-ebfcb90f06ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:40.422042 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bac09-9c7b-447b-ab62-ebfcb90f06ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:40.424495 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bac09-9c7b-447b-ab62-ebfcb90f06ec service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:40.441340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1404/2814] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:34:40 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:34:40.551692 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ce162-6ae0-498b-98b3-15143265b3b0 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:40.563737 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ce162-6ae0-498b-98b3-15143265b3b0 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:40.568608 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476d40d-4044-408f-8d78-607732c17611 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:40.569246 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476d40d-4044-408f-8d78-607732c17611 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:40.569527 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476d40d-4044-408f-8d78-607732c17611 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:40.569801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476d40d-4044-408f-8d78-607732c17611 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:40.647121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476d40d-4044-408f-8d78-607732c17611 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:40.649970 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476d40d-4044-408f-8d78-607732c17611 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:40.667923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1411/2815] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:34:40 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:34:45.135481 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bac09-9c7b-447b-ab62-ebfcb90f06ec service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:45.217340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:45.238664 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bac09-9c7b-447b-ab62-ebfcb90f06ec service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:45.247221 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cbad1-6738-4824-949b-f45028716341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:45.248236 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cbad1-6738-4824-949b-f45028716341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:45.248663 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cbad1-6738-4824-949b-f45028716341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:45.249082 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cbad1-6738-4824-949b-f45028716341 admin admin] PATH_INFO: `/v3/users/45d9a8c4c0ea4b3b8aa38351ff55edd4` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:45.250895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cbad1-6738-4824-949b-f45028716341 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45d9a8c4c0ea4b3b8aa38351ff55edd4)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:45.253419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cbad1-6738-4824-949b-f45028716341 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:45.327063 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6e9cbad1-6738-4824-949b-f45028716341 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45d9a8c4c0ea4b3b8aa38351ff55edd4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:45.336122 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6e9cbad1-6738-4824-949b-f45028716341 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45d9a8c4c0ea4b3b8aa38351ff55edd4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:45.357876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6e9cbad1-6738-4824-949b-f45028716341 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45d9a8c4c0ea4b3b8aa38351ff55edd4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:45.358755 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6e9cbad1-6738-4824-949b-f45028716341 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45d9a8c4c0ea4b3b8aa38351ff55edd4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:45.371152 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-6e9cbad1-6738-4824-949b-f45028716341 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45d9a8c4c0ea4b3b8aa38351ff55edd4'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:45.379267 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1405/2816] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:34:45 2022] DELETE /identity/v3/users/45d9a8c4c0ea4b3b8aa38351ff55edd4 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:34:45.383790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe68e5-1d65-4cce-b243-aaaf871ecb87 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:45.384646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe68e5-1d65-4cce-b243-aaaf871ecb87 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:45.384646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe68e5-1d65-4cce-b243-aaaf871ecb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:45.407321 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-bbbe68e5-1d65-4cce-b243-aaaf871ecb87 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:45.408916 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-bbbe68e5-1d65-4cce-b243-aaaf871ecb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:45.485756 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1412/2817] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:34:45.496984 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cbad1-6738-4824-949b-f45028716341 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:45.501860 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-6e9cbad1-6738-4824-949b-f45028716341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:45.566008 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cbad1-6738-4824-949b-f45028716341 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:45.569851 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5eb8c-d2f1-4622-8a3f-275f45dc4c0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:45.570480 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5eb8c-d2f1-4622-8a3f-275f45dc4c0a service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:45.570762 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5eb8c-d2f1-4622-8a3f-275f45dc4c0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:45.571029 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5eb8c-d2f1-4622-8a3f-275f45dc4c0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:45.581291 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5eb8c-d2f1-4622-8a3f-275f45dc4c0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:45.582931 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5eb8c-d2f1-4622-8a3f-275f45dc4c0a service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:45.593972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1406/2818] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:34:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:34:45.905067 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe68e5-1d65-4cce-b243-aaaf871ecb87 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:45.907155 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-bbbe68e5-1d65-4cce-b243-aaaf871ecb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:45.922469 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe68e5-1d65-4cce-b243-aaaf871ecb87 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:45.929290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7036ce-076c-4486-9764-60785a8d2111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:45.930075 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7036ce-076c-4486-9764-60785a8d2111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:45.930360 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7036ce-076c-4486-9764-60785a8d2111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:45.930641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7036ce-076c-4486-9764-60785a8d2111 admin admin] PATH_INFO: `/v3/projects/e961955456a84871b2a100ab368872f8` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:45.932187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7036ce-076c-4486-9764-60785a8d2111 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e961955456a84871b2a100ab368872f8)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:45.934374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7036ce-076c-4486-9764-60785a8d2111 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:45.990746 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-aa7036ce-076c-4486-9764-60785a8d2111 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e961955456a84871b2a100ab368872f8', 'request_id': 'req-aa7036ce-076c-4486-9764-60785a8d2111'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:45.992131 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:46.000747 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-aa7036ce-076c-4486-9764-60785a8d2111 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e961955456a84871b2a100ab368872f8', 'request_id': 'req-aa7036ce-076c-4486-9764-60785a8d2111'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:46.011931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-aa7036ce-076c-4486-9764-60785a8d2111 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e961955456a84871b2a100ab368872f8', 'request_id': 'req-aa7036ce-076c-4486-9764-60785a8d2111'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:46.029364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-aa7036ce-076c-4486-9764-60785a8d2111 admin admin] The token cache is being invalidate because project e961955456a84871b2a100ab368872f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:34:46.029734 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-aa7036ce-076c-4486-9764-60785a8d2111 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:46.030465 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-aa7036ce-076c-4486-9764-60785a8d2111 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:46.032153 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1413/2819] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:34:45 2022] DELETE /identity/v3/projects/e961955456a84871b2a100ab368872f8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:34:46.036852 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5eb8c-d2f1-4622-8a3f-275f45dc4c0a service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:46.048564 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5eb8c-d2f1-4622-8a3f-275f45dc4c0a service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:46.120948 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266353ab-6977-4c83-8691-a5971d16197d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:46.121610 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266353ab-6977-4c83-8691-a5971d16197d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:46.121891 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266353ab-6977-4c83-8691-a5971d16197d admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:46.122166 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266353ab-6977-4c83-8691-a5971d16197d admin admin] PATH_INFO: `/v3/users/145fa08085944dc8bd9bcfe93ff1a0c2` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:46.123374 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266353ab-6977-4c83-8691-a5971d16197d admin admin] RBAC: Authorizing `identity:delete_user(user_id=145fa08085944dc8bd9bcfe93ff1a0c2)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:46.125408 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266353ab-6977-4c83-8691-a5971d16197d admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:46.147378 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:46.189141 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-266353ab-6977-4c83-8691-a5971d16197d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '145fa08085944dc8bd9bcfe93ff1a0c2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:46.198305 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-266353ab-6977-4c83-8691-a5971d16197d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '145fa08085944dc8bd9bcfe93ff1a0c2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:46.213501 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-266353ab-6977-4c83-8691-a5971d16197d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '145fa08085944dc8bd9bcfe93ff1a0c2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:46.214397 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-266353ab-6977-4c83-8691-a5971d16197d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '145fa08085944dc8bd9bcfe93ff1a0c2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:46.223758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-266353ab-6977-4c83-8691-a5971d16197d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '145fa08085944dc8bd9bcfe93ff1a0c2'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:46.231295 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1407/2820] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:34:46 2022] DELETE /identity/v3/users/145fa08085944dc8bd9bcfe93ff1a0c2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:34:46.634311 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7036ce-076c-4486-9764-60785a8d2111 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:46.636253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-aa7036ce-076c-4486-9764-60785a8d2111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:46.650945 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7036ce-076c-4486-9764-60785a8d2111 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:46.736873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38971719-77dc-46d1-b673-b05477019c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:46.737606 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38971719-77dc-46d1-b673-b05477019c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:46.737891 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38971719-77dc-46d1-b673-b05477019c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:46.738169 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38971719-77dc-46d1-b673-b05477019c5b admin admin] PATH_INFO: `/v3/projects/b5d07db194d448768d84efab1c6bebcf` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:46.739533 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38971719-77dc-46d1-b673-b05477019c5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5d07db194d448768d84efab1c6bebcf)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:46.741931 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38971719-77dc-46d1-b673-b05477019c5b admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:46.800953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-38971719-77dc-46d1-b673-b05477019c5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5d07db194d448768d84efab1c6bebcf', 'request_id': 'req-38971719-77dc-46d1-b673-b05477019c5b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:46.813357 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-38971719-77dc-46d1-b673-b05477019c5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5d07db194d448768d84efab1c6bebcf', 'request_id': 'req-38971719-77dc-46d1-b673-b05477019c5b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:46.827145 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-38971719-77dc-46d1-b673-b05477019c5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5d07db194d448768d84efab1c6bebcf', 'request_id': 'req-38971719-77dc-46d1-b673-b05477019c5b'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:46.843462 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-38971719-77dc-46d1-b673-b05477019c5b admin admin] The token cache is being invalidate because project b5d07db194d448768d84efab1c6bebcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:34:46.843914 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-38971719-77dc-46d1-b673-b05477019c5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:46.844624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-38971719-77dc-46d1-b673-b05477019c5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:46.846180 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1414/2821] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:34:46 2022] DELETE /identity/v3/projects/b5d07db194d448768d84efab1c6bebcf => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:34:46.851421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02cbc4-cfe9-4a47-92d8-903903036548 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:46.851721 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02cbc4-cfe9-4a47-92d8-903903036548 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:46.853048 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e02cbc4-cfe9-4a47-92d8-903903036548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:46.874249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-1e02cbc4-cfe9-4a47-92d8-903903036548 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:46.875581 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-1e02cbc4-cfe9-4a47-92d8-903903036548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:46.954741 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1408/2822] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:34:46.963018 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b213a-7d17-4fd6-909b-29006386309d None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:46.963382 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b213a-7d17-4fd6-909b-29006386309d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:46.963908 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b213a-7d17-4fd6-909b-29006386309d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:46.989295 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-355b213a-7d17-4fd6-909b-29006386309d None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:46.990652 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-355b213a-7d17-4fd6-909b-29006386309d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:46.999672 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1415/2823] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:34:47.005056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02cbc4-cfe9-4a47-92d8-903903036548 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:47.022114 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e02cbc4-cfe9-4a47-92d8-903903036548 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:47.028257 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b89165e-67c1-44a3-9c5d-188eccf0976d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:47.028898 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89165e-67c1-44a3-9c5d-188eccf0976d admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:47.029173 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89165e-67c1-44a3-9c5d-188eccf0976d admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:47.029439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89165e-67c1-44a3-9c5d-188eccf0976d admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:47.030139 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b89165e-67c1-44a3-9c5d-188eccf0976d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:34:47.030437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b89165e-67c1-44a3-9c5d-188eccf0976d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:47.032142 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b89165e-67c1-44a3-9c5d-188eccf0976d admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:47.048105 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1409/2824] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:34:47.052794 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b213a-7d17-4fd6-909b-29006386309d None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:47.064936 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b213a-7d17-4fd6-909b-29006386309d None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:47.069850 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea449ce-beec-4b8e-ba06-7ff523f27348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:47.070469 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea449ce-beec-4b8e-ba06-7ff523f27348 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:47.070816 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea449ce-beec-4b8e-ba06-7ff523f27348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:47.071006 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea449ce-beec-4b8e-ba06-7ff523f27348 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:47.071703 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea449ce-beec-4b8e-ba06-7ff523f27348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:47.073618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea449ce-beec-4b8e-ba06-7ff523f27348 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:47.118503 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1416/2825] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:34:47 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:34:47.119744 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b89165e-67c1-44a3-9c5d-188eccf0976d admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:47.130820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b89165e-67c1-44a3-9c5d-188eccf0976d admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:47.201707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affaecd-ca5b-4e42-9dbd-ed6eb827cd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:47.202344 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affaecd-ca5b-4e42-9dbd-ed6eb827cd91 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:47.202640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affaecd-ca5b-4e42-9dbd-ed6eb827cd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:47.202914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affaecd-ca5b-4e42-9dbd-ed6eb827cd91 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:47.203556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affaecd-ca5b-4e42-9dbd-ed6eb827cd91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:47.205604 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affaecd-ca5b-4e42-9dbd-ed6eb827cd91 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:47.230531 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1410/2826] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:34:47 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:34:47.234923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea449ce-beec-4b8e-ba06-7ff523f27348 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:47.250265 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea449ce-beec-4b8e-ba06-7ff523f27348 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:47.256321 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37258ab1-3fd9-4403-aca0-40d336c47054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:47.256974 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37258ab1-3fd9-4403-aca0-40d336c47054 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:47.257249 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37258ab1-3fd9-4403-aca0-40d336c47054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:47.257524 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37258ab1-3fd9-4403-aca0-40d336c47054 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:47.258199 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37258ab1-3fd9-4403-aca0-40d336c47054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:47.259912 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37258ab1-3fd9-4403-aca0-40d336c47054 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:47.276172 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1417/2827] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:34:47.278164 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affaecd-ca5b-4e42-9dbd-ed6eb827cd91 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:47.289475 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affaecd-ca5b-4e42-9dbd-ed6eb827cd91 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:47.295153 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e2513-98fe-400b-b331-b475e7bd7019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:47.295803 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e2513-98fe-400b-b331-b475e7bd7019 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:47.296155 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e2513-98fe-400b-b331-b475e7bd7019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:47.296441 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e2513-98fe-400b-b331-b475e7bd7019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:47.297116 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71e2513-98fe-400b-b331-b475e7bd7019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:47.299461 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71e2513-98fe-400b-b331-b475e7bd7019 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:47.306207 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:47.314882 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1411/2828] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:34:47.318760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37258ab1-3fd9-4403-aca0-40d336c47054 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:47.339544 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37258ab1-3fd9-4403-aca0-40d336c47054 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:47.348156 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1242375-b367-4a95-8ac2-ca2d2c7cd713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:47.349242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1242375-b367-4a95-8ac2-ca2d2c7cd713 admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:47.349436 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1242375-b367-4a95-8ac2-ca2d2c7cd713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:47.349703 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1242375-b367-4a95-8ac2-ca2d2c7cd713 admin admin] PATH_INFO: `/v3/projects/738fa212c0dc449e84672f6727a705c4/users/41952aea8f8944dda74cb49bb8105fde/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:47.375338 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1242375-b367-4a95-8ac2-ca2d2c7cd713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=738fa212c0dc449e84672f6727a705c4, user_id=41952aea8f8944dda74cb49bb8105fde, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:47.380913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1242375-b367-4a95-8ac2-ca2d2c7cd713 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:47.404388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1418/2829] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:34:47 2022] PUT /identity/v3/projects/738fa212c0dc449e84672f6727a705c4/users/41952aea8f8944dda74cb49bb8105fde/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:34:47.420480 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312557a-35ac-404c-86f9-7d672deb2e99 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:47.420948 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312557a-35ac-404c-86f9-7d672deb2e99 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:47.421422 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312557a-35ac-404c-86f9-7d672deb2e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:47.458047 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-1312557a-35ac-404c-86f9-7d672deb2e99 None None] MFA Rules not processed for user `41952aea8f8944dda74cb49bb8105fde`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:47.459330 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-1312557a-35ac-404c-86f9-7d672deb2e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:47.554189 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1412/2830] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:34:47.566287 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1242375-b367-4a95-8ac2-ca2d2c7cd713 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:47.568255 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-c1242375-b367-4a95-8ac2-ca2d2c7cd713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:47.576946 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:47.656788 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1242375-b367-4a95-8ac2-ca2d2c7cd713 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:47.661911 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0107da-ef5f-4995-a17c-40aa897073e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:47.662458 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0107da-ef5f-4995-a17c-40aa897073e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:47.662858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0107da-ef5f-4995-a17c-40aa897073e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:47.663262 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0107da-ef5f-4995-a17c-40aa897073e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:47.675046 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0107da-ef5f-4995-a17c-40aa897073e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:47.676573 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0107da-ef5f-4995-a17c-40aa897073e2 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:47.687086 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1419/2831] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:34:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:34:55.581091 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312557a-35ac-404c-86f9-7d672deb2e99 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:55.665765 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312557a-35ac-404c-86f9-7d672deb2e99 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:55.670842 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972aa883-58a1-401a-abc5-4e5b5f5c8c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:55.671627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972aa883-58a1-401a-abc5-4e5b5f5c8c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:55.672018 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972aa883-58a1-401a-abc5-4e5b5f5c8c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:55.672396 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972aa883-58a1-401a-abc5-4e5b5f5c8c6f admin admin] PATH_INFO: `/v3/users/41952aea8f8944dda74cb49bb8105fde` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:55.673682 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972aa883-58a1-401a-abc5-4e5b5f5c8c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=41952aea8f8944dda74cb49bb8105fde)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:55.675708 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972aa883-58a1-401a-abc5-4e5b5f5c8c6f admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:55.732820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-972aa883-58a1-401a-abc5-4e5b5f5c8c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41952aea8f8944dda74cb49bb8105fde'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:55.745553 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-972aa883-58a1-401a-abc5-4e5b5f5c8c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41952aea8f8944dda74cb49bb8105fde'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:55.770555 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-972aa883-58a1-401a-abc5-4e5b5f5c8c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41952aea8f8944dda74cb49bb8105fde'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:55.771745 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-972aa883-58a1-401a-abc5-4e5b5f5c8c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41952aea8f8944dda74cb49bb8105fde'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:55.785434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-972aa883-58a1-401a-abc5-4e5b5f5c8c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41952aea8f8944dda74cb49bb8105fde'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:55.796178 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1413/2832] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:34:55 2022] DELETE /identity/v3/users/41952aea8f8944dda74cb49bb8105fde => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:34:55.802474 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98080fdd-d966-4df8-b29e-6fbca54ee015 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:55.803224 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98080fdd-d966-4df8-b29e-6fbca54ee015 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:55.803224 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98080fdd-d966-4df8-b29e-6fbca54ee015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:55.826640 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-98080fdd-d966-4df8-b29e-6fbca54ee015 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:55.828350 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-98080fdd-d966-4df8-b29e-6fbca54ee015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:55.904810 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1420/2833] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:34:55.918080 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972aa883-58a1-401a-abc5-4e5b5f5c8c6f admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:55.920925 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-972aa883-58a1-401a-abc5-4e5b5f5c8c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:55.941808 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:56.012659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972aa883-58a1-401a-abc5-4e5b5f5c8c6f admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:56.017696 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998dde3-b951-417d-8118-26a57596bc21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:56.018390 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998dde3-b951-417d-8118-26a57596bc21 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:56.018672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998dde3-b951-417d-8118-26a57596bc21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:56.018946 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998dde3-b951-417d-8118-26a57596bc21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:56.030015 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998dde3-b951-417d-8118-26a57596bc21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:56.031633 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998dde3-b951-417d-8118-26a57596bc21 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:56.043230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1414/2834] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:34:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:34:56.447582 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98080fdd-d966-4df8-b29e-6fbca54ee015 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:56.449458 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-98080fdd-d966-4df8-b29e-6fbca54ee015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:56.468307 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98080fdd-d966-4df8-b29e-6fbca54ee015 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:56.474325 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be0359f-915b-4f0d-a2b6-74f188492933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:56.475491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0359f-915b-4f0d-a2b6-74f188492933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:56.475798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0359f-915b-4f0d-a2b6-74f188492933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:56.476293 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be0359f-915b-4f0d-a2b6-74f188492933 admin admin] PATH_INFO: `/v3/projects/738fa212c0dc449e84672f6727a705c4` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:56.478027 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0359f-915b-4f0d-a2b6-74f188492933 admin admin] RBAC: Authorizing `identity:delete_project(project_id=738fa212c0dc449e84672f6727a705c4)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:56.481023 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be0359f-915b-4f0d-a2b6-74f188492933 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:56.543269 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-3be0359f-915b-4f0d-a2b6-74f188492933 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '738fa212c0dc449e84672f6727a705c4', 'request_id': 'req-3be0359f-915b-4f0d-a2b6-74f188492933'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:56.551585 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-3be0359f-915b-4f0d-a2b6-74f188492933 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '738fa212c0dc449e84672f6727a705c4', 'request_id': 'req-3be0359f-915b-4f0d-a2b6-74f188492933'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:56.561513 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-3be0359f-915b-4f0d-a2b6-74f188492933 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '738fa212c0dc449e84672f6727a705c4', 'request_id': 'req-3be0359f-915b-4f0d-a2b6-74f188492933'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:56.588069 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-3be0359f-915b-4f0d-a2b6-74f188492933 admin admin] The token cache is being invalidate because project 738fa212c0dc449e84672f6727a705c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:34:56.588364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-3be0359f-915b-4f0d-a2b6-74f188492933 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:56.589282 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-3be0359f-915b-4f0d-a2b6-74f188492933 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:34:56.591189 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1421/2835] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:34:56 2022] DELETE /identity/v3/projects/738fa212c0dc449e84672f6727a705c4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:34:56.597950 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd751b5-af6c-48b9-9572-6927455d4e1b None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:56.598573 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd751b5-af6c-48b9-9572-6927455d4e1b None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:56.598885 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd751b5-af6c-48b9-9572-6927455d4e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:56.632704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-5cd751b5-af6c-48b9-9572-6927455d4e1b None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:56.635013 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-5cd751b5-af6c-48b9-9572-6927455d4e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:56.714501 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1415/2836] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:34:56.723064 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ff7d0-8a82-4613-9508-18e0f2092a80 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:56.723489 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ff7d0-8a82-4613-9508-18e0f2092a80 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:56.723846 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1ff7d0-8a82-4613-9508-18e0f2092a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:56.748467 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-ea1ff7d0-8a82-4613-9508-18e0f2092a80 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:56.750226 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ea1ff7d0-8a82-4613-9508-18e0f2092a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:56.759388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1422/2837] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:34:56.764535 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd751b5-af6c-48b9-9572-6927455d4e1b None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:56.775656 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd751b5-af6c-48b9-9572-6927455d4e1b None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:56.781335 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41918340-c817-492c-a876-a22c51eef155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:56.782137 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41918340-c817-492c-a876-a22c51eef155 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:56.782540 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41918340-c817-492c-a876-a22c51eef155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:56.782984 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41918340-c817-492c-a876-a22c51eef155 admin admin] PATH_INFO: `/v3/domains` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:56.783778 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41918340-c817-492c-a876-a22c51eef155 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80138) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:34:56.784200 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41918340-c817-492c-a876-a22c51eef155 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:56.785917 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41918340-c817-492c-a876-a22c51eef155 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:56.800929 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1416/2838] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:34:56.806046 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ff7d0-8a82-4613-9508-18e0f2092a80 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:56.816617 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1ff7d0-8a82-4613-9508-18e0f2092a80 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:56.822838 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cff2b0-605f-4d0b-8fb8-5527e4a75204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:56.823890 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cff2b0-605f-4d0b-8fb8-5527e4a75204 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:56.824413 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cff2b0-605f-4d0b-8fb8-5527e4a75204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:56.824883 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cff2b0-605f-4d0b-8fb8-5527e4a75204 admin admin] PATH_INFO: `/v3/projects` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:56.826122 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cff2b0-605f-4d0b-8fb8-5527e4a75204 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:56.829809 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cff2b0-605f-4d0b-8fb8-5527e4a75204 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:56.870763 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1423/2839] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:34:56 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:34:56.878394 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41918340-c817-492c-a876-a22c51eef155 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:56.890508 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41918340-c817-492c-a876-a22c51eef155 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:56.962291 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da7dc6-15eb-4d81-bf55-ce1b3caa0fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:56.962930 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da7dc6-15eb-4d81-bf55-ce1b3caa0fb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:56.963216 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da7dc6-15eb-4d81-bf55-ce1b3caa0fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:56.963483 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1da7dc6-15eb-4d81-bf55-ce1b3caa0fb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:56.964171 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da7dc6-15eb-4d81-bf55-ce1b3caa0fb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:56.966259 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1da7dc6-15eb-4d81-bf55-ce1b3caa0fb8 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:56.998988 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1417/2840] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:34:56 2022] POST /identity/v3/users => generated 286 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:34:57.003897 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cff2b0-605f-4d0b-8fb8-5527e4a75204 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:57.017253 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:57.029664 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cff2b0-605f-4d0b-8fb8-5527e4a75204 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:57.038866 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe7051-b092-47f4-b132-785281eaf661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:57.039873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe7051-b092-47f4-b132-785281eaf661 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:57.040411 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe7051-b092-47f4-b132-785281eaf661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:57.040874 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe7051-b092-47f4-b132-785281eaf661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:57.042091 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe7051-b092-47f4-b132-785281eaf661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:57.045617 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe7051-b092-47f4-b132-785281eaf661 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:57.067484 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1424/2841] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:34:57.071828 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da7dc6-15eb-4d81-bf55-ce1b3caa0fb8 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:57.082980 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1da7dc6-15eb-4d81-bf55-ce1b3caa0fb8 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:57.088223 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe8080-22b9-459e-baf4-9528c022b6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:57.088867 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe8080-22b9-459e-baf4-9528c022b6a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:57.089149 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe8080-22b9-459e-baf4-9528c022b6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:57.089421 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe8080-22b9-459e-baf4-9528c022b6a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:57.090087 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe8080-22b9-459e-baf4-9528c022b6a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:57.091751 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe8080-22b9-459e-baf4-9528c022b6a9 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:57.103437 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1418/2842] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:34:57.108532 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe7051-b092-47f4-b132-785281eaf661 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:57.120359 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe7051-b092-47f4-b132-785281eaf661 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:57.126801 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ee49b-9a11-4058-97de-dc79f56be7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:57.127679 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ee49b-9a11-4058-97de-dc79f56be7ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:57.128066 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ee49b-9a11-4058-97de-dc79f56be7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:57.128385 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ee49b-9a11-4058-97de-dc79f56be7ca admin admin] PATH_INFO: `/v3/projects/a5467695b424429fb3ef05e2c2b6bccc/users/f5c6bd8aee50465793a4d99ba3b310ce/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:57.148198 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ee49b-9a11-4058-97de-dc79f56be7ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5467695b424429fb3ef05e2c2b6bccc, user_id=f5c6bd8aee50465793a4d99ba3b310ce, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:57.150944 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ee49b-9a11-4058-97de-dc79f56be7ca admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:57.167914 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1425/2843] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:34:57 2022] PUT /identity/v3/projects/a5467695b424429fb3ef05e2c2b6bccc/users/f5c6bd8aee50465793a4d99ba3b310ce/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:34:57.180650 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb7934-583a-4bcb-82f4-fef1adffac9c None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:57.181101 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb7934-583a-4bcb-82f4-fef1adffac9c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:57.181492 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb7934-583a-4bcb-82f4-fef1adffac9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:57.220345 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-4ffb7934-583a-4bcb-82f4-fef1adffac9c None None] MFA Rules not processed for user `f5c6bd8aee50465793a4d99ba3b310ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:34:57.222237 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4ffb7934-583a-4bcb-82f4-fef1adffac9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:57.230903 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:34:57.303717 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1419/2844] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:34:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:34:57.316271 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ee49b-9a11-4058-97de-dc79f56be7ca admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:57.318807 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-6a6ee49b-9a11-4058-97de-dc79f56be7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:57.386497 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ee49b-9a11-4058-97de-dc79f56be7ca admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:57.390576 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae007d9c-33bc-4cb5-87b7-bdef8aaed5a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:57.391205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae007d9c-33bc-4cb5-87b7-bdef8aaed5a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:57.391481 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae007d9c-33bc-4cb5-87b7-bdef8aaed5a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:57.391750 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae007d9c-33bc-4cb5-87b7-bdef8aaed5a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:57.402255 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae007d9c-33bc-4cb5-87b7-bdef8aaed5a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:57.404047 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae007d9c-33bc-4cb5-87b7-bdef8aaed5a8 service cinder] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:57.421550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1426/2845] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:34:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:34:59.589722 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb7934-583a-4bcb-82f4-fef1adffac9c None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:34:59.591846 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-4ffb7934-583a-4bcb-82f4-fef1adffac9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:59.669555 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb7934-583a-4bcb-82f4-fef1adffac9c None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:34:59.673604 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93816c4d-e23b-47bb-abea-03562118c8c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:34:59.674243 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93816c4d-e23b-47bb-abea-03562118c8c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:34:59.674550 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93816c4d-e23b-47bb-abea-03562118c8c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:34:59.674988 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93816c4d-e23b-47bb-abea-03562118c8c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:34:59.677008 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-93816c4d-e23b-47bb-abea-03562118c8c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:34:59.761837 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93816c4d-e23b-47bb-abea-03562118c8c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:34:59.763704 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93816c4d-e23b-47bb-abea-03562118c8c6 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:34:59.777630 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1420/2846] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:34:59 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:35:00.861248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae007d9c-33bc-4cb5-87b7-bdef8aaed5a8 service cinder] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:00.941611 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae007d9c-33bc-4cb5-87b7-bdef8aaed5a8 service cinder] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:00.947774 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa279ab9-9334-4fc8-b345-ef07f60fe543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:00.948567 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa279ab9-9334-4fc8-b345-ef07f60fe543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:00.948913 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa279ab9-9334-4fc8-b345-ef07f60fe543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:00.949258 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa279ab9-9334-4fc8-b345-ef07f60fe543 admin admin] PATH_INFO: `/v3/users/f5c6bd8aee50465793a4d99ba3b310ce` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:00.950553 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa279ab9-9334-4fc8-b345-ef07f60fe543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5c6bd8aee50465793a4d99ba3b310ce)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:00.952961 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa279ab9-9334-4fc8-b345-ef07f60fe543 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:01.035593 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-aa279ab9-9334-4fc8-b345-ef07f60fe543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5c6bd8aee50465793a4d99ba3b310ce'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:01.059352 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-aa279ab9-9334-4fc8-b345-ef07f60fe543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5c6bd8aee50465793a4d99ba3b310ce'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:01.073745 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:35:01.077813 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-aa279ab9-9334-4fc8-b345-ef07f60fe543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5c6bd8aee50465793a4d99ba3b310ce'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:01.079204 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-aa279ab9-9334-4fc8-b345-ef07f60fe543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5c6bd8aee50465793a4d99ba3b310ce'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:01.080450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-aa279ab9-9334-4fc8-b345-ef07f60fe543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5c6bd8aee50465793a4d99ba3b310ce'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:01.108604 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1427/2847] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:35:00 2022] DELETE /identity/v3/users/f5c6bd8aee50465793a4d99ba3b310ce => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:35:01.115715 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfd9eb-c07f-4203-a8e7-7f1999699650 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:01.116588 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfd9eb-c07f-4203-a8e7-7f1999699650 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:01.116804 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfd9eb-c07f-4203-a8e7-7f1999699650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:01.149510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-27bfd9eb-c07f-4203-a8e7-7f1999699650 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:35:01.151653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-27bfd9eb-c07f-4203-a8e7-7f1999699650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:35:01.251404 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:35:01.268957 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1421/2848] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:35:01.284591 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa279ab9-9334-4fc8-b345-ef07f60fe543 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:01.286481 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-aa279ab9-9334-4fc8-b345-ef07f60fe543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:35:01.356092 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa279ab9-9334-4fc8-b345-ef07f60fe543 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:01.360506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc69b81-1cce-46a7-b6cd-118b7ec8434c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:01.361831 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc69b81-1cce-46a7-b6cd-118b7ec8434c service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:01.361831 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc69b81-1cce-46a7-b6cd-118b7ec8434c service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:01.362298 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc69b81-1cce-46a7-b6cd-118b7ec8434c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:01.376117 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc69b81-1cce-46a7-b6cd-118b7ec8434c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:01.377975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc69b81-1cce-46a7-b6cd-118b7ec8434c service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:01.389647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1428/2849] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:35:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:35:01.738009 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfd9eb-c07f-4203-a8e7-7f1999699650 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:01.739625 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-27bfd9eb-c07f-4203-a8e7-7f1999699650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:35:01.754083 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfd9eb-c07f-4203-a8e7-7f1999699650 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:01.759205 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0f4ac-9c3d-4186-9ae8-e4af2b4a121c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:01.760294 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0f4ac-9c3d-4186-9ae8-e4af2b4a121c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:01.760427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0f4ac-9c3d-4186-9ae8-e4af2b4a121c admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:01.760627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0f4ac-9c3d-4186-9ae8-e4af2b4a121c admin admin] PATH_INFO: `/v3/projects/a5467695b424429fb3ef05e2c2b6bccc` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:01.761897 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0f4ac-9c3d-4186-9ae8-e4af2b4a121c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5467695b424429fb3ef05e2c2b6bccc)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:01.763854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0f4ac-9c3d-4186-9ae8-e4af2b4a121c admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:01.835730 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-90b0f4ac-9c3d-4186-9ae8-e4af2b4a121c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5467695b424429fb3ef05e2c2b6bccc', 'request_id': 'req-90b0f4ac-9c3d-4186-9ae8-e4af2b4a121c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:01.847113 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-90b0f4ac-9c3d-4186-9ae8-e4af2b4a121c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5467695b424429fb3ef05e2c2b6bccc', 'request_id': 'req-90b0f4ac-9c3d-4186-9ae8-e4af2b4a121c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:01.854889 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-90b0f4ac-9c3d-4186-9ae8-e4af2b4a121c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5467695b424429fb3ef05e2c2b6bccc', 'request_id': 'req-90b0f4ac-9c3d-4186-9ae8-e4af2b4a121c'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:01.877301 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-90b0f4ac-9c3d-4186-9ae8-e4af2b4a121c admin admin] The token cache is being invalidate because project a5467695b424429fb3ef05e2c2b6bccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:35:01.877562 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-90b0f4ac-9c3d-4186-9ae8-e4af2b4a121c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:01.879706 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-90b0f4ac-9c3d-4186-9ae8-e4af2b4a121c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:01.881653 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1422/2850] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:35:01 2022] DELETE /identity/v3/projects/a5467695b424429fb3ef05e2c2b6bccc => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:35:01.891780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6e959-e8a6-4977-be4d-f90afe79e184 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:01.892565 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6e959-e8a6-4977-be4d-f90afe79e184 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:01.892764 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6e959-e8a6-4977-be4d-f90afe79e184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:01.921912 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-87e6e959-e8a6-4977-be4d-f90afe79e184 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:35:01.923406 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-87e6e959-e8a6-4977-be4d-f90afe79e184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:35:01.990704 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1429/2851] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:35:01.999248 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b451853-ab6b-4b21-b916-630f277b632c None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:01.999847 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b451853-ab6b-4b21-b916-630f277b632c None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:02.000256 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b451853-ab6b-4b21-b916-630f277b632c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:02.036406 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-9b451853-ab6b-4b21-b916-630f277b632c None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:35:02.038959 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-9b451853-ab6b-4b21-b916-630f277b632c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:35:02.047346 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1423/2852] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:35:02.052678 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6e959-e8a6-4977-be4d-f90afe79e184 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:02.078177 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6e959-e8a6-4977-be4d-f90afe79e184 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:02.091290 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fe057-c832-4de1-b2fe-191fe2304f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:02.092595 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fe057-c832-4de1-b2fe-191fe2304f60 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:02.093221 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fe057-c832-4de1-b2fe-191fe2304f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:02.093804 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fe057-c832-4de1-b2fe-191fe2304f60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:02.094753 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fe057-c832-4de1-b2fe-191fe2304f60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:35:02.095417 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fe057-c832-4de1-b2fe-191fe2304f60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:02.099240 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fe057-c832-4de1-b2fe-191fe2304f60 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:02.122989 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1430/2853] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 19:35:02.124719 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b451853-ab6b-4b21-b916-630f277b632c None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:02.136876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b451853-ab6b-4b21-b916-630f277b632c None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:02.142788 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0b3b8-1f0c-415e-90c9-552c0c7c5a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:02.143867 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0b3b8-1f0c-415e-90c9-552c0c7c5a14 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:02.144209 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0b3b8-1f0c-415e-90c9-552c0c7c5a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:02.144487 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0b3b8-1f0c-415e-90c9-552c0c7c5a14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:02.145158 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c0b3b8-1f0c-415e-90c9-552c0c7c5a14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:02.147258 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c0b3b8-1f0c-415e-90c9-552c0c7c5a14 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:02.191943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1424/2854] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:35:02 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:35:02.198383 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fe057-c832-4de1-b2fe-191fe2304f60 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:02.219181 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fe057-c832-4de1-b2fe-191fe2304f60 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:02.373741 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158cc98-fbc1-49c3-b570-b4121650e36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:02.374932 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158cc98-fbc1-49c3-b570-b4121650e36e admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:02.375977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158cc98-fbc1-49c3-b570-b4121650e36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:02.376453 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158cc98-fbc1-49c3-b570-b4121650e36e admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:02.378344 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158cc98-fbc1-49c3-b570-b4121650e36e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:02.383148 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158cc98-fbc1-49c3-b570-b4121650e36e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:02.429631 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1431/2855] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:35:02 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:35:02.436627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0b3b8-1f0c-415e-90c9-552c0c7c5a14 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:02.457642 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c0b3b8-1f0c-415e-90c9-552c0c7c5a14 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:02.492401 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3a429-f2b2-4db8-a6e8-7befe9bd4eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:02.493823 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3a429-f2b2-4db8-a6e8-7befe9bd4eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:02.494460 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3a429-f2b2-4db8-a6e8-7befe9bd4eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:02.495186 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3a429-f2b2-4db8-a6e8-7befe9bd4eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:02.500246 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3a429-f2b2-4db8-a6e8-7befe9bd4eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:02.506712 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3a429-f2b2-4db8-a6e8-7befe9bd4eb5 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:02.550664 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1425/2856] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:35:02.554893 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158cc98-fbc1-49c3-b570-b4121650e36e admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:02.577780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158cc98-fbc1-49c3-b570-b4121650e36e admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:02.590242 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9b26c-e111-47ed-9502-edd30494481d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:02.591537 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9b26c-e111-47ed-9502-edd30494481d admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:02.592158 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9b26c-e111-47ed-9502-edd30494481d admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:02.592574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9b26c-e111-47ed-9502-edd30494481d admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:02.594088 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9b26c-e111-47ed-9502-edd30494481d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:02.597217 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9b26c-e111-47ed-9502-edd30494481d admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:02.600762 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:35:02.624081 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1432/2857] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:35:02.630592 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3a429-f2b2-4db8-a6e8-7befe9bd4eb5 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:02.653397 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3a429-f2b2-4db8-a6e8-7befe9bd4eb5 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:02.665686 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799eda43-d6e5-432e-ad58-3a88b7fb3268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:02.666875 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799eda43-d6e5-432e-ad58-3a88b7fb3268 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:02.667444 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799eda43-d6e5-432e-ad58-3a88b7fb3268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:02.668316 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799eda43-d6e5-432e-ad58-3a88b7fb3268 admin admin] PATH_INFO: `/v3/projects/7b3deca935894797bf15da83243ae125/users/cd28831fc5994eceb77f107186e084c6/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:02.696839 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799eda43-d6e5-432e-ad58-3a88b7fb3268 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3deca935894797bf15da83243ae125, user_id=cd28831fc5994eceb77f107186e084c6, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:02.701598 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799eda43-d6e5-432e-ad58-3a88b7fb3268 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:02.731808 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1426/2858] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:35:02 2022] PUT /identity/v3/projects/7b3deca935894797bf15da83243ae125/users/cd28831fc5994eceb77f107186e084c6/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:35:02.744585 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65202864-3505-4b9b-a9e8-218772121098 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:02.746141 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65202864-3505-4b9b-a9e8-218772121098 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:02.746631 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65202864-3505-4b9b-a9e8-218772121098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:02.796179 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-65202864-3505-4b9b-a9e8-218772121098 None None] MFA Rules not processed for user `cd28831fc5994eceb77f107186e084c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:35:02.798121 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-65202864-3505-4b9b-a9e8-218772121098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:35:02.883766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1433/2859] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:35:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:35:02.893555 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799eda43-d6e5-432e-ad58-3a88b7fb3268 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:02.895858 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-799eda43-d6e5-432e-ad58-3a88b7fb3268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:35:03.002357 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799eda43-d6e5-432e-ad58-3a88b7fb3268 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:03.014770 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50a437-232a-4dce-ba08-bdd15b6ca01f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a9daa1a3c7654dcfa5ccb03f467ce7d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:03.015638 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50a437-232a-4dce-ba08-bdd15b6ca01f service glance] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:03.015876 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50a437-232a-4dce-ba08-bdd15b6ca01f service glance] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:03.016315 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50a437-232a-4dce-ba08-bdd15b6ca01f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:03.033127 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50a437-232a-4dce-ba08-bdd15b6ca01f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:03.035013 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50a437-232a-4dce-ba08-bdd15b6ca01f service glance] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:03.049203 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1427/2860] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:35:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:35:12.876588 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=33 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:35:13.043524 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:35:29.792647 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65202864-3505-4b9b-a9e8-218772121098 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:29.869129 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65202864-3505-4b9b-a9e8-218772121098 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:29.874474 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f3a82-a7f5-4560-bb8a-daf1c93a366e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:29.875144 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f3a82-a7f5-4560-bb8a-daf1c93a366e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:29.875424 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f3a82-a7f5-4560-bb8a-daf1c93a366e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:29.875701 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f3a82-a7f5-4560-bb8a-daf1c93a366e admin admin] PATH_INFO: `/v3/users/cd28831fc5994eceb77f107186e084c6` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:29.877024 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f3a82-a7f5-4560-bb8a-daf1c93a366e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd28831fc5994eceb77f107186e084c6)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:29.879012 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f3a82-a7f5-4560-bb8a-daf1c93a366e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:29.939982 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f31f3a82-a7f5-4560-bb8a-daf1c93a366e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd28831fc5994eceb77f107186e084c6'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:29.954649 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f31f3a82-a7f5-4560-bb8a-daf1c93a366e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd28831fc5994eceb77f107186e084c6'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:29.964224 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f31f3a82-a7f5-4560-bb8a-daf1c93a366e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd28831fc5994eceb77f107186e084c6'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:29.965187 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f31f3a82-a7f5-4560-bb8a-daf1c93a366e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd28831fc5994eceb77f107186e084c6'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:29.965954 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f31f3a82-a7f5-4560-bb8a-daf1c93a366e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd28831fc5994eceb77f107186e084c6'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:29.982843 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1434/2861] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:35:29 2022] DELETE /identity/v3/users/cd28831fc5994eceb77f107186e084c6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:35:29.989289 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa9826-c174-4554-a5ef-affb5ed0bb8a None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:29.989956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa9826-c174-4554-a5ef-affb5ed0bb8a None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:29.989956 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affa9826-c174-4554-a5ef-affb5ed0bb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:30.013389 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-affa9826-c174-4554-a5ef-affb5ed0bb8a None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:35:30.015731 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-affa9826-c174-4554-a5ef-affb5ed0bb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:35:30.113651 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1428/2862] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:35:30.142916 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f3a82-a7f5-4560-bb8a-daf1c93a366e admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:30.144811 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f31f3a82-a7f5-4560-bb8a-daf1c93a366e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:35:30.239223 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f3a82-a7f5-4560-bb8a-daf1c93a366e admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:30.245506 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef38e0c-80f4-4dec-ac8a-5aca6c35f24d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:30.246480 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef38e0c-80f4-4dec-ac8a-5aca6c35f24d service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:30.246891 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef38e0c-80f4-4dec-ac8a-5aca6c35f24d service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:30.247292 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef38e0c-80f4-4dec-ac8a-5aca6c35f24d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:30.258806 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef38e0c-80f4-4dec-ac8a-5aca6c35f24d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:30.260822 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef38e0c-80f4-4dec-ac8a-5aca6c35f24d service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:30.273573 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1435/2863] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:35:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:35:30.615353 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa9826-c174-4554-a5ef-affb5ed0bb8a None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:30.616916 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-affa9826-c174-4554-a5ef-affb5ed0bb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:35:30.630849 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affa9826-c174-4554-a5ef-affb5ed0bb8a None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:30.637024 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cec40d-d8e3-4e32-89df-37dbd23deffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:30.637784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cec40d-d8e3-4e32-89df-37dbd23deffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:30.637996 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cec40d-d8e3-4e32-89df-37dbd23deffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:30.638347 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cec40d-d8e3-4e32-89df-37dbd23deffe admin admin] PATH_INFO: `/v3/projects/7b3deca935894797bf15da83243ae125` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:30.639621 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cec40d-d8e3-4e32-89df-37dbd23deffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b3deca935894797bf15da83243ae125)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:30.641680 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cec40d-d8e3-4e32-89df-37dbd23deffe admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:30.713940 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-00cec40d-d8e3-4e32-89df-37dbd23deffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b3deca935894797bf15da83243ae125', 'request_id': 'req-00cec40d-d8e3-4e32-89df-37dbd23deffe'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:30.720703 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-00cec40d-d8e3-4e32-89df-37dbd23deffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b3deca935894797bf15da83243ae125', 'request_id': 'req-00cec40d-d8e3-4e32-89df-37dbd23deffe'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:30.728621 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-00cec40d-d8e3-4e32-89df-37dbd23deffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b3deca935894797bf15da83243ae125', 'request_id': 'req-00cec40d-d8e3-4e32-89df-37dbd23deffe'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:30.748089 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-00cec40d-d8e3-4e32-89df-37dbd23deffe admin admin] The token cache is being invalidate because project 7b3deca935894797bf15da83243ae125 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:35:30.748587 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-00cec40d-d8e3-4e32-89df-37dbd23deffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:30.749750 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-00cec40d-d8e3-4e32-89df-37dbd23deffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:35:30.752439 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1429/2864] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:35:30 2022] DELETE /identity/v3/projects/7b3deca935894797bf15da83243ae125 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 19:35:30.759376 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafd377-f74b-4ac7-ad46-840472f1f19e None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:30.759783 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafd377-f74b-4ac7-ad46-840472f1f19e None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:30.760169 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafd377-f74b-4ac7-ad46-840472f1f19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:30.783476 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-daafd377-f74b-4ac7-ad46-840472f1f19e None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:35:30.785083 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-daafd377-f74b-4ac7-ad46-840472f1f19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:35:30.813381 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:35:30.895162 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1436/2865] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:35:30.993949 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170249c8-f569-4456-af24-8ae4ab7a2708 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:30.994782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170249c8-f569-4456-af24-8ae4ab7a2708 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:30.994782 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170249c8-f569-4456-af24-8ae4ab7a2708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:31.019632 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-170249c8-f569-4456-af24-8ae4ab7a2708 None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:35:31.025627 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-170249c8-f569-4456-af24-8ae4ab7a2708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:35:31.034636 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1430/2866] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:35:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:35:31.039620 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafd377-f74b-4ac7-ad46-840472f1f19e None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:31.050780 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafd377-f74b-4ac7-ad46-840472f1f19e None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:31.058753 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcca800-5c7e-4664-8d45-ebadb77a6941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:31.059480 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcca800-5c7e-4664-8d45-ebadb77a6941 admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:31.060585 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcca800-5c7e-4664-8d45-ebadb77a6941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:31.061586 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcca800-5c7e-4664-8d45-ebadb77a6941 admin admin] PATH_INFO: `/v3/domains` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:31.064043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcca800-5c7e-4664-8d45-ebadb77a6941 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80139) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 19:35:31.070364 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcca800-5c7e-4664-8d45-ebadb77a6941 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:31.072740 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcca800-5c7e-4664-8d45-ebadb77a6941 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:31.087464 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1437/2867] 10.222.0.17 () {58 vars in 1153 bytes} [Sun Aug 7 19:35:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 19:35:31.095067 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170249c8-f569-4456-af24-8ae4ab7a2708 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:31.108844 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170249c8-f569-4456-af24-8ae4ab7a2708 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:31.115880 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c30c24-9788-4abd-9dd9-45ff0d77d317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:31.116870 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c30c24-9788-4abd-9dd9-45ff0d77d317 admin admin] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:31.117293 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c30c24-9788-4abd-9dd9-45ff0d77d317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:31.117692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c30c24-9788-4abd-9dd9-45ff0d77d317 admin admin] PATH_INFO: `/v3/projects` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:31.118652 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c30c24-9788-4abd-9dd9-45ff0d77d317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:31.121679 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c30c24-9788-4abd-9dd9-45ff0d77d317 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:31.162478 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1431/2868] 10.222.0.17 () {60 vars in 1153 bytes} [Sun Aug 7 19:35:31 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 19:35:31.168176 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcca800-5c7e-4664-8d45-ebadb77a6941 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:31.182744 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcca800-5c7e-4664-8d45-ebadb77a6941 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:31.252572 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94975f-388f-4974-ab9d-fd0e609f9df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:31.253287 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94975f-388f-4974-ab9d-fd0e609f9df8 admin admin] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:31.253648 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94975f-388f-4974-ab9d-fd0e609f9df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:31.254043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94975f-388f-4974-ab9d-fd0e609f9df8 admin admin] PATH_INFO: `/v3/users` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:31.254739 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94975f-388f-4974-ab9d-fd0e609f9df8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:31.256915 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94975f-388f-4974-ab9d-fd0e609f9df8 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:31.282557 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1438/2869] 10.222.0.17 () {60 vars in 1144 bytes} [Sun Aug 7 19:35:31 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 19:35:31.287412 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c30c24-9788-4abd-9dd9-45ff0d77d317 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:31.299526 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c30c24-9788-4abd-9dd9-45ff0d77d317 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:31.306142 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544be16-bade-48bc-9f62-09eab2d94016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:31.307000 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544be16-bade-48bc-9f62-09eab2d94016 admin admin] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:31.309594 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544be16-bade-48bc-9f62-09eab2d94016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:31.310208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544be16-bade-48bc-9f62-09eab2d94016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:31.311047 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544be16-bade-48bc-9f62-09eab2d94016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:31.313008 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544be16-bade-48bc-9f62-09eab2d94016 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:31.331625 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1432/2870] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 19:35:31.337446 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94975f-388f-4974-ab9d-fd0e609f9df8 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:31.349638 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94975f-388f-4974-ab9d-fd0e609f9df8 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:31.356123 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ccbed-ed71-4e48-9b12-6fca05b950ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:31.356766 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ccbed-ed71-4e48-9b12-6fca05b950ba admin admin] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:31.357049 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ccbed-ed71-4e48-9b12-6fca05b950ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:31.357323 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ccbed-ed71-4e48-9b12-6fca05b950ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:31.357999 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ccbed-ed71-4e48-9b12-6fca05b950ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:31.359677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96ccbed-ed71-4e48-9b12-6fca05b950ba admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:31.374485 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1439/2871] 10.222.0.17 () {58 vars in 1122 bytes} [Sun Aug 7 19:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 19:35:31.381419 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544be16-bade-48bc-9f62-09eab2d94016 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:31.399838 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544be16-bade-48bc-9f62-09eab2d94016 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:31.408115 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09a09f-fb56-4547-ae02-973bd02a8059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:31.409417 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09a09f-fb56-4547-ae02-973bd02a8059 admin admin] REQUEST_METHOD: `PUT` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:31.409900 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09a09f-fb56-4547-ae02-973bd02a8059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:31.410376 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09a09f-fb56-4547-ae02-973bd02a8059 admin admin] PATH_INFO: `/v3/projects/af90b9767e934addbde86fbe26c72c0a/users/d8077e19cce64faab6e5d71e3e56880d/roles/246bbd6878334bba893bc6f5bf5caee0` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:31.436923 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09a09f-fb56-4547-ae02-973bd02a8059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af90b9767e934addbde86fbe26c72c0a, user_id=d8077e19cce64faab6e5d71e3e56880d, role_id=246bbd6878334bba893bc6f5bf5caee0)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:31.441758 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c09a09f-fb56-4547-ae02-973bd02a8059 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:31.465659 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1433/2872] 10.222.0.17 () {60 vars in 1483 bytes} [Sun Aug 7 19:35:31 2022] PUT /identity/v3/projects/af90b9767e934addbde86fbe26c72c0a/users/d8077e19cce64faab6e5d71e3e56880d/roles/246bbd6878334bba893bc6f5bf5caee0 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:35:31.472362 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4b2c9-d3cc-4ba3-8679-76944d4bc03d None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:31.472884 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4b2c9-d3cc-4ba3-8679-76944d4bc03d None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:31.473272 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a4b2c9-d3cc-4ba3-8679-76944d4bc03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:31.498463 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-d9a4b2c9-d3cc-4ba3-8679-76944d4bc03d None None] MFA Rules not processed for user `39e61fe22f50447d975033301fd5b706`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:35:31.500076 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d9a4b2c9-d3cc-4ba3-8679-76944d4bc03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:35:31.582979 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1440/2873] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:35:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 19:35:31.602360 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09a09f-fb56-4547-ae02-973bd02a8059 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:31.605001 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-3c09a09f-fb56-4547-ae02-973bd02a8059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:35:31.642616 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:35:31.693152 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09a09f-fb56-4547-ae02-973bd02a8059 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:31.697491 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e225c-7cbc-4394-b38a-ee1ff4985e72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:31.698256 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e225c-7cbc-4394-b38a-ee1ff4985e72 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:31.698647 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e225c-7cbc-4394-b38a-ee1ff4985e72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:31.699065 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e225c-7cbc-4394-b38a-ee1ff4985e72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:31.710226 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e225c-7cbc-4394-b38a-ee1ff4985e72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:31.711963 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e225c-7cbc-4394-b38a-ee1ff4985e72 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:31.723760 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1434/2874] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:35:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:35:32.578525 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b9b63-6e82-4cd2-937a-4df54dd46347 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:32.579499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b9b63-6e82-4cd2-937a-4df54dd46347 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:32.579499 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b9b63-6e82-4cd2-937a-4df54dd46347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:32.630388 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-383b9b63-6e82-4cd2-937a-4df54dd46347 None None] MFA Rules not processed for user `d8077e19cce64faab6e5d71e3e56880d`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:35:32.632865 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-383b9b63-6e82-4cd2-937a-4df54dd46347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:35:32.673532 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:35:32.758435 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1441/2875] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:35:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:35:32.775087 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e225c-7cbc-4394-b38a-ee1ff4985e72 service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:35:32.778340 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-5d2e225c-7cbc-4394-b38a-ee1ff4985e72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:35:32.877181 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e225c-7cbc-4394-b38a-ee1ff4985e72 service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:35:32.883328 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492a2e1-634a-406b-80b9-92ee51545314 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:35:32.884111 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492a2e1-634a-406b-80b9-92ee51545314 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:32.884394 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492a2e1-634a-406b-80b9-92ee51545314 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:32.884670 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492a2e1-634a-406b-80b9-92ee51545314 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:32.896905 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492a2e1-634a-406b-80b9-92ee51545314 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:35:32.898409 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492a2e1-634a-406b-80b9-92ee51545314 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:35:32.910858 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1435/2876] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:35:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:35:34.394589 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59aaa-bf84-43e9-8381-3bf7d8df06a0 None None] REQUEST_METHOD: `POST` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:35:34.395519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59aaa-bf84-43e9-8381-3bf7d8df06a0 None None] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:35:34.395519 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59aaa-bf84-43e9-8381-3bf7d8df06a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:35:34.417289 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.auth.core [None req-d0f59aaa-bf84-43e9-8381-3bf7d8df06a0 None None] MFA Rules not processed for user `d8077e19cce64faab6e5d71e3e56880d`. Rule list: `[]` (Enabled: `True`). {{(pid=80139) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:35:34.418533 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-d0f59aaa-bf84-43e9-8381-3bf7d8df06a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:35:34.428505 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1442/2877] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:35:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:35:42.906258 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=43 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:35:44.411641 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=22 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:36:00.974081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492a2e1-634a-406b-80b9-92ee51545314 service cinder] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:36:00.990311 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492a2e1-634a-406b-80b9-92ee51545314 service cinder] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:36:00.997056 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a789a-d5e2-4b64-a079-e399c10bfdf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f83e5f0b1d5e4f57bbf283d420f080ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:36:00.998093 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a789a-d5e2-4b64-a079-e399c10bfdf5 service cinder] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:36:00.998535 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a789a-d5e2-4b64-a079-e399c10bfdf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:36:00.999032 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a789a-d5e2-4b64-a079-e399c10bfdf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:36:01.001992 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-5f1a789a-d5e2-4b64-a079-e399c10bfdf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:01.104620 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a789a-d5e2-4b64-a079-e399c10bfdf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:36:01.106016 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a789a-d5e2-4b64-a079-e399c10bfdf5 service cinder] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:36:01.131098 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1436/2878] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:36:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:36:11.127809 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=43 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:36:15.731270 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f59aaa-bf84-43e9-8381-3bf7d8df06a0 None None] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:36:15.745588 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f59aaa-bf84-43e9-8381-3bf7d8df06a0 None None] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:36:15.750905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98282607-fd9f-415e-b746-0610da6dadb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:36:15.751624 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98282607-fd9f-415e-b746-0610da6dadb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:36:15.751951 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98282607-fd9f-415e-b746-0610da6dadb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:36:15.752318 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98282607-fd9f-415e-b746-0610da6dadb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:36:15.754460 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-98282607-fd9f-415e-b746-0610da6dadb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:15.770953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98282607-fd9f-415e-b746-0610da6dadb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:36:15.772661 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98282607-fd9f-415e-b746-0610da6dadb6 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:36:15.788280 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1443/2879] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:36:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:36:25.782177 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=6 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:36:28.835405 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780eb702-2fea-4fe2-875f-c1af01fb3e91 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:36:28.836066 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780eb702-2fea-4fe2-875f-c1af01fb3e91 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:36:28.836066 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780eb702-2fea-4fe2-875f-c1af01fb3e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:36:28.911072 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-780eb702-2fea-4fe2-875f-c1af01fb3e91 None None] MFA Rules not processed for user `61ff29258a214817b8fc15b51fe8b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:36:28.915710 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-780eb702-2fea-4fe2-875f-c1af01fb3e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:28.998598 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1437/2880] 10.222.0.17 () {58 vars in 967 bytes} [Sun Aug 7 19:36:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:36:29.048219 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98282607-fd9f-415e-b746-0610da6dadb6 service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:36:29.049823 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-98282607-fd9f-415e-b746-0610da6dadb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:29.155718 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98282607-fd9f-415e-b746-0610da6dadb6 service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:36:29.162111 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de6cfa-7328-48a0-8292-e41ec7bbc4e6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db51971158a946c98a8ed716ecc5d72b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:36:29.162760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de6cfa-7328-48a0-8292-e41ec7bbc4e6 service swift] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:36:29.163039 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de6cfa-7328-48a0-8292-e41ec7bbc4e6 service swift] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:36:29.163311 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de6cfa-7328-48a0-8292-e41ec7bbc4e6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:36:29.174029 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de6cfa-7328-48a0-8292-e41ec7bbc4e6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:36:29.175790 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9de6cfa-7328-48a0-8292-e41ec7bbc4e6 service swift] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:36:29.185703 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1444/2881] 10.222.0.17 () {60 vars in 1384 bytes} [Sun Aug 7 19:36:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 19:36:29.317601 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f52e23-b837-413f-9730-fac5155edb62 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:36:29.318082 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f52e23-b837-413f-9730-fac5155edb62 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:36:29.318555 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f52e23-b837-413f-9730-fac5155edb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:36:29.356850 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-34f52e23-b837-413f-9730-fac5155edb62 None None] MFA Rules not processed for user `656a586e718543bfb095790441c28fe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:36:29.358304 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-34f52e23-b837-413f-9730-fac5155edb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:29.441001 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1438/2882] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:36:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:36:29.965208 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de6cfa-7328-48a0-8292-e41ec7bbc4e6 service swift] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:36:29.978798 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de6cfa-7328-48a0-8292-e41ec7bbc4e6 service swift] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:36:29.984864 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbd22e-2c45-4d6c-a839-f8a01e279671 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:36:29.985737 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbd22e-2c45-4d6c-a839-f8a01e279671 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:36:29.986234 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbd22e-2c45-4d6c-a839-f8a01e279671 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:36:29.986694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbd22e-2c45-4d6c-a839-f8a01e279671 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:36:29.989907 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-64dbd22e-2c45-4d6c-a839-f8a01e279671 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:30.003235 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbd22e-2c45-4d6c-a839-f8a01e279671 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:36:30.005927 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbd22e-2c45-4d6c-a839-f8a01e279671 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:36:30.033898 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1445/2883] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:36:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:36:30.886301 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f52e23-b837-413f-9730-fac5155edb62 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:36:30.887935 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-34f52e23-b837-413f-9730-fac5155edb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:30.989972 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f52e23-b837-413f-9730-fac5155edb62 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:36:30.999002 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e6ec34-a8e0-47e4-bdf5-cb6922a84298 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '143c30b4201645279564e72ab3288adc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:36:31.000120 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6ec34-a8e0-47e4-bdf5-cb6922a84298 service placement] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:36:31.000611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6ec34-a8e0-47e4-bdf5-cb6922a84298 service placement] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:36:31.001092 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e6ec34-a8e0-47e4-bdf5-cb6922a84298 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:36:31.004983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-24e6ec34-a8e0-47e4-bdf5-cb6922a84298 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:31.041907 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6ec34-a8e0-47e4-bdf5-cb6922a84298 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:36:31.044377 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e6ec34-a8e0-47e4-bdf5-cb6922a84298 service placement] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:36:31.057597 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1439/2884] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:36:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:36:35.305752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbd22e-2c45-4d6c-a839-f8a01e279671 service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:36:35.308205 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-64dbd22e-2c45-4d6c-a839-f8a01e279671 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:35.325895 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbd22e-2c45-4d6c-a839-f8a01e279671 service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:36:35.330468 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c6cdd-dea1-430a-b100-4020da4fba7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:36:35.331108 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c6cdd-dea1-430a-b100-4020da4fba7f service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:36:35.331392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c6cdd-dea1-430a-b100-4020da4fba7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:36:35.331665 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c6cdd-dea1-430a-b100-4020da4fba7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:36:35.333638 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9e2c6cdd-dea1-430a-b100-4020da4fba7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:35.355677 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c6cdd-dea1-430a-b100-4020da4fba7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:36:35.358826 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c6cdd-dea1-430a-b100-4020da4fba7f service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:36:35.375623 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1446/2885] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:36:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:36:38.283307 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497897e-4eb9-4b93-b318-37685176d1e3 None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:36:38.283862 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497897e-4eb9-4b93-b318-37685176d1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:36:38.284362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497897e-4eb9-4b93-b318-37685176d1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:36:38.307539 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=100 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:36:38.320206 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-6497897e-4eb9-4b93-b318-37685176d1e3 None None] MFA Rules not processed for user `d8077e19cce64faab6e5d71e3e56880d`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:36:38.324341 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-6497897e-4eb9-4b93-b318-37685176d1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:38.333914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1440/2886] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:36:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:36:38.617523 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c6cdd-dea1-430a-b100-4020da4fba7f service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:36:38.627758 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c6cdd-dea1-430a-b100-4020da4fba7f service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:36:38.631863 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911ea73-a65a-407e-a564-666b530b7f44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:36:38.632573 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911ea73-a65a-407e-a564-666b530b7f44 service neutron] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:36:38.632807 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911ea73-a65a-407e-a564-666b530b7f44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:36:38.633392 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911ea73-a65a-407e-a564-666b530b7f44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:36:38.637314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-1911ea73-a65a-407e-a564-666b530b7f44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:38.649170 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911ea73-a65a-407e-a564-666b530b7f44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:36:38.650957 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911ea73-a65a-407e-a564-666b530b7f44 service neutron] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:36:38.663900 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1447/2887] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:36:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:36:38.680742 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497897e-4eb9-4b93-b318-37685176d1e3 None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:36:38.684580 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-6497897e-4eb9-4b93-b318-37685176d1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:38.715202 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497897e-4eb9-4b93-b318-37685176d1e3 None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:36:38.722892 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8af22-6f29-4d17-a44b-a9c80e5a2ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:36:38.723977 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8af22-6f29-4d17-a44b-a9c80e5a2ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:36:38.724533 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8af22-6f29-4d17-a44b-a9c80e5a2ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:36:38.725032 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8af22-6f29-4d17-a44b-a9c80e5a2ef8 admin admin] PATH_INFO: `/v3/users/523936e9a8ca49b9805fa110a1841d64` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:36:38.727707 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8af22-6f29-4d17-a44b-a9c80e5a2ef8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=523936e9a8ca49b9805fa110a1841d64)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:36:38.732161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8af22-6f29-4d17-a44b-a9c80e5a2ef8 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:36:38.805051 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-8ab8af22-6f29-4d17-a44b-a9c80e5a2ef8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '523936e9a8ca49b9805fa110a1841d64'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:38.812913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-8ab8af22-6f29-4d17-a44b-a9c80e5a2ef8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '523936e9a8ca49b9805fa110a1841d64'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:38.829727 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-8ab8af22-6f29-4d17-a44b-a9c80e5a2ef8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '523936e9a8ca49b9805fa110a1841d64'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:38.830841 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-8ab8af22-6f29-4d17-a44b-a9c80e5a2ef8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '523936e9a8ca49b9805fa110a1841d64'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:38.841483 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-8ab8af22-6f29-4d17-a44b-a9c80e5a2ef8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '523936e9a8ca49b9805fa110a1841d64'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:38.853037 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1441/2888] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:36:38 2022] DELETE /identity/v3/users/523936e9a8ca49b9805fa110a1841d64 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:36:39.168410 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911ea73-a65a-407e-a564-666b530b7f44 service neutron] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:36:39.191862 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911ea73-a65a-407e-a564-666b530b7f44 service neutron] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:36:39.272356 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475b6df-acfe-4fb3-b3bb-831d0a7da962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:36:39.273011 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475b6df-acfe-4fb3-b3bb-831d0a7da962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:36:39.273280 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475b6df-acfe-4fb3-b3bb-831d0a7da962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:36:39.273760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475b6df-acfe-4fb3-b3bb-831d0a7da962 admin admin] PATH_INFO: `/v3/projects/1b16920c2e384c9ca20252917204b8ab` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:36:39.274933 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475b6df-acfe-4fb3-b3bb-831d0a7da962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b16920c2e384c9ca20252917204b8ab)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:36:39.276876 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475b6df-acfe-4fb3-b3bb-831d0a7da962 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:36:39.316297 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:36:39.337954 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-9475b6df-acfe-4fb3-b3bb-831d0a7da962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b16920c2e384c9ca20252917204b8ab', 'request_id': 'req-9475b6df-acfe-4fb3-b3bb-831d0a7da962'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:39.350745 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-9475b6df-acfe-4fb3-b3bb-831d0a7da962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b16920c2e384c9ca20252917204b8ab', 'request_id': 'req-9475b6df-acfe-4fb3-b3bb-831d0a7da962'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:39.361351 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-9475b6df-acfe-4fb3-b3bb-831d0a7da962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b16920c2e384c9ca20252917204b8ab', 'request_id': 'req-9475b6df-acfe-4fb3-b3bb-831d0a7da962'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:39.376078 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-9475b6df-acfe-4fb3-b3bb-831d0a7da962 admin admin] The token cache is being invalidate because project 1b16920c2e384c9ca20252917204b8ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:36:39.376379 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-9475b6df-acfe-4fb3-b3bb-831d0a7da962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:39.377274 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-9475b6df-acfe-4fb3-b3bb-831d0a7da962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:39.379367 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1448/2889] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:36:39 2022] DELETE /identity/v3/projects/1b16920c2e384c9ca20252917204b8ab => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:36:39.383548 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8af22-6f29-4d17-a44b-a9c80e5a2ef8 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:36:39.385216 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-8ab8af22-6f29-4d17-a44b-a9c80e5a2ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:39.399631 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8af22-6f29-4d17-a44b-a9c80e5a2ef8 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:36:39.497307 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e47e6-a48a-4a0a-8558-157e95949066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:36:39.497943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e47e6-a48a-4a0a-8558-157e95949066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:36:39.498463 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e47e6-a48a-4a0a-8558-157e95949066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:36:39.499211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e47e6-a48a-4a0a-8558-157e95949066 admin admin] PATH_INFO: `/v3/users/98e071c1b74b4e03b46789d7f75dc0aa` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:36:39.501462 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e47e6-a48a-4a0a-8558-157e95949066 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98e071c1b74b4e03b46789d7f75dc0aa)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:36:39.505602 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6e47e6-a48a-4a0a-8558-157e95949066 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:36:39.578382 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2d6e47e6-a48a-4a0a-8558-157e95949066 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98e071c1b74b4e03b46789d7f75dc0aa'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:39.591760 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2d6e47e6-a48a-4a0a-8558-157e95949066 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98e071c1b74b4e03b46789d7f75dc0aa'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:39.605725 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2d6e47e6-a48a-4a0a-8558-157e95949066 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98e071c1b74b4e03b46789d7f75dc0aa'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:39.606711 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2d6e47e6-a48a-4a0a-8558-157e95949066 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98e071c1b74b4e03b46789d7f75dc0aa'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:39.608646 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:36:39.618816 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-2d6e47e6-a48a-4a0a-8558-157e95949066 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98e071c1b74b4e03b46789d7f75dc0aa'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:39.636613 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1442/2890] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:36:39 2022] DELETE /identity/v3/users/98e071c1b74b4e03b46789d7f75dc0aa => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:36:39.862161 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475b6df-acfe-4fb3-b3bb-831d0a7da962 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:36:39.864942 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9475b6df-acfe-4fb3-b3bb-831d0a7da962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:39.893486 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475b6df-acfe-4fb3-b3bb-831d0a7da962 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:36:39.964726 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c446bc-787f-4871-b9fd-1b4de1526ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:36:39.965439 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c446bc-787f-4871-b9fd-1b4de1526ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:36:39.965929 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c446bc-787f-4871-b9fd-1b4de1526ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:36:39.966330 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c446bc-787f-4871-b9fd-1b4de1526ab9 admin admin] PATH_INFO: `/v3/projects/c1f2aa38ad0248cb9661edc469252361` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:36:39.967590 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c446bc-787f-4871-b9fd-1b4de1526ab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f2aa38ad0248cb9661edc469252361)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:36:39.969581 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c446bc-787f-4871-b9fd-1b4de1526ab9 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:36:40.024198 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-26c446bc-787f-4871-b9fd-1b4de1526ab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f2aa38ad0248cb9661edc469252361', 'request_id': 'req-26c446bc-787f-4871-b9fd-1b4de1526ab9'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:40.030447 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-26c446bc-787f-4871-b9fd-1b4de1526ab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f2aa38ad0248cb9661edc469252361', 'request_id': 'req-26c446bc-787f-4871-b9fd-1b4de1526ab9'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:40.039475 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-26c446bc-787f-4871-b9fd-1b4de1526ab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f2aa38ad0248cb9661edc469252361', 'request_id': 'req-26c446bc-787f-4871-b9fd-1b4de1526ab9'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:40.050977 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-26c446bc-787f-4871-b9fd-1b4de1526ab9 admin admin] The token cache is being invalidate because project c1f2aa38ad0248cb9661edc469252361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:36:40.051251 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-26c446bc-787f-4871-b9fd-1b4de1526ab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:40.051921 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-26c446bc-787f-4871-b9fd-1b4de1526ab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:36:40.053466 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1449/2891] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:36:39 2022] DELETE /identity/v3/projects/c1f2aa38ad0248cb9661edc469252361 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:36:42.077828 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e47e6-a48a-4a0a-8558-157e95949066 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:36:42.079824 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-2d6e47e6-a48a-4a0a-8558-157e95949066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:42.181230 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e47e6-a48a-4a0a-8558-157e95949066 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:36:42.253456 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0765b-861e-4b1f-b648-e67a2f2529e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:36:42.254084 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0765b-861e-4b1f-b648-e67a2f2529e3 service nova] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:36:42.254362 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0765b-861e-4b1f-b648-e67a2f2529e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:36:42.254635 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0765b-861e-4b1f-b648-e67a2f2529e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:36:42.256692 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b8c0765b-861e-4b1f-b648-e67a2f2529e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:42.271223 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0765b-861e-4b1f-b648-e67a2f2529e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:36:42.273070 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0765b-861e-4b1f-b648-e67a2f2529e3 service nova] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:36:42.286240 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1443/2892] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:36:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:36:50.047692 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=63,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:36:52.281398 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=64 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:36:52.903572 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c446bc-787f-4871-b9fd-1b4de1526ab9 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:36:52.905151 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-26c446bc-787f-4871-b9fd-1b4de1526ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:52.923314 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c446bc-787f-4871-b9fd-1b4de1526ab9 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:36:52.929449 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0736c-dca4-4cbd-9351-5257f4313be0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:36:52.930090 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0736c-dca4-4cbd-9351-5257f4313be0 service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:36:52.930370 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0736c-dca4-4cbd-9351-5257f4313be0 service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:36:52.930646 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0736c-dca4-4cbd-9351-5257f4313be0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:36:52.933153 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-e4c0736c-dca4-4cbd-9351-5257f4313be0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:36:52.953708 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c0736c-dca4-4cbd-9351-5257f4313be0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:36:52.958011 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c0736c-dca4-4cbd-9351-5257f4313be0 service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:36:52.972618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1450/2893] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:36:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:37:02.967026 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=6 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:37:33.450640 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0765b-861e-4b1f-b648-e67a2f2529e3 service nova] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:37:33.452225 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-b8c0765b-861e-4b1f-b648-e67a2f2529e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:37:33.524434 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0765b-861e-4b1f-b648-e67a2f2529e3 service nova] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:37:33.530090 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f0872-7a4a-48e6-a342-3892ba45a826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:37:33.530895 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f0872-7a4a-48e6-a342-3892ba45a826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:37:33.531182 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f0872-7a4a-48e6-a342-3892ba45a826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:37:33.531521 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f0872-7a4a-48e6-a342-3892ba45a826 admin admin] PATH_INFO: `/v3/users/d8077e19cce64faab6e5d71e3e56880d` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:37:33.532913 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f0872-7a4a-48e6-a342-3892ba45a826 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8077e19cce64faab6e5d71e3e56880d)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:37:33.534957 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f0872-7a4a-48e6-a342-3892ba45a826 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:37:33.589232 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-032f0872-7a4a-48e6-a342-3892ba45a826 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8077e19cce64faab6e5d71e3e56880d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:37:33.601854 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-032f0872-7a4a-48e6-a342-3892ba45a826 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8077e19cce64faab6e5d71e3e56880d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:37:33.617208 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-032f0872-7a4a-48e6-a342-3892ba45a826 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8077e19cce64faab6e5d71e3e56880d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:37:33.618043 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-032f0872-7a4a-48e6-a342-3892ba45a826 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8077e19cce64faab6e5d71e3e56880d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:37:33.629059 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-032f0872-7a4a-48e6-a342-3892ba45a826 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8077e19cce64faab6e5d71e3e56880d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:37:33.635974 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1444/2894] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:37:33 2022] DELETE /identity/v3/users/d8077e19cce64faab6e5d71e3e56880d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:37:33.836947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0736c-dca4-4cbd-9351-5257f4313be0 service nova] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:37:33.855760 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0736c-dca4-4cbd-9351-5257f4313be0 service nova] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:37:33.929915 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cc28b-13e5-42c4-af0e-4b455040a75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:37:33.930550 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cc28b-13e5-42c4-af0e-4b455040a75e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:37:33.930833 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cc28b-13e5-42c4-af0e-4b455040a75e admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:37:33.931132 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cc28b-13e5-42c4-af0e-4b455040a75e admin admin] PATH_INFO: `/v3/projects/af90b9767e934addbde86fbe26c72c0a` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:37:33.932374 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cc28b-13e5-42c4-af0e-4b455040a75e admin admin] RBAC: Authorizing `identity:delete_project(project_id=af90b9767e934addbde86fbe26c72c0a)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:37:33.934311 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cc28b-13e5-42c4-af0e-4b455040a75e admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:37:33.993567 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f19cc28b-13e5-42c4-af0e-4b455040a75e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af90b9767e934addbde86fbe26c72c0a', 'request_id': 'req-f19cc28b-13e5-42c4-af0e-4b455040a75e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:37:34.002501 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f19cc28b-13e5-42c4-af0e-4b455040a75e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af90b9767e934addbde86fbe26c72c0a', 'request_id': 'req-f19cc28b-13e5-42c4-af0e-4b455040a75e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:37:34.014260 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f19cc28b-13e5-42c4-af0e-4b455040a75e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af90b9767e934addbde86fbe26c72c0a', 'request_id': 'req-f19cc28b-13e5-42c4-af0e-4b455040a75e'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:37:34.026429 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f19cc28b-13e5-42c4-af0e-4b455040a75e admin admin] The token cache is being invalidate because project af90b9767e934addbde86fbe26c72c0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80139) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:37:34.026905 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f19cc28b-13e5-42c4-af0e-4b455040a75e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:37:34.027792 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f19cc28b-13e5-42c4-af0e-4b455040a75e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:37:34.029953 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1451/2895] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:37:33 2022] DELETE /identity/v3/projects/af90b9767e934addbde86fbe26c72c0a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:37:43.631983 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:37:44.022574 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:38:29.386081 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6b406-d06d-4bd0-897a-a1e274bc38ca None None] REQUEST_METHOD: `POST` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:38:29.386732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6b406-d06d-4bd0-897a-a1e274bc38ca None None] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:38:29.386732 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6b406-d06d-4bd0-897a-a1e274bc38ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:38:29.407527 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.auth.core [None req-a9f6b406-d06d-4bd0-897a-a1e274bc38ca None None] MFA Rules not processed for user `656a586e718543bfb095790441c28fe8`. Rule list: `[]` (Enabled: `True`). {{(pid=80138) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 19:38:29.408455 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a9f6b406-d06d-4bd0-897a-a1e274bc38ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:38:29.470599 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1445/2896] 10.222.0.17 () {58 vars in 958 bytes} [Sun Aug 7 19:38:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 19:38:39.463176 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=25 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:38:42.611567 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cc28b-13e5-42c4-af0e-4b455040a75e admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:38:42.613322 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f19cc28b-13e5-42c4-af0e-4b455040a75e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:38:42.703221 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cc28b-13e5-42c4-af0e-4b455040a75e admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:38:42.709694 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fa5f6-6f81-4489-a844-93b7e573ca99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4946aec299f45bd8402cff3159d04d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:38:42.710335 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fa5f6-6f81-4489-a844-93b7e573ca99 service nova] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:38:42.710618 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fa5f6-6f81-4489-a844-93b7e573ca99 service nova] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:38:42.710923 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fa5f6-6f81-4489-a844-93b7e573ca99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:38:42.712826 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ec3fa5f6-6f81-4489-a844-93b7e573ca99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:38:42.726081 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3fa5f6-6f81-4489-a844-93b7e573ca99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:38:42.727672 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3fa5f6-6f81-4489-a844-93b7e573ca99 service nova] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:38:42.739341 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1452/2897] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:38:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:38:52.734360 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=34 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:40:16.936625 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6b406-d06d-4bd0-897a-a1e274bc38ca None None] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:40:16.938116 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-a9f6b406-d06d-4bd0-897a-a1e274bc38ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:40:17.005161 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6b406-d06d-4bd0-897a-a1e274bc38ca None None] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:40:17.009381 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974a58f-e710-4587-b1a4-b87b38f91062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:40:17.009991 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974a58f-e710-4587-b1a4-b87b38f91062 service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:40:17.010281 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974a58f-e710-4587-b1a4-b87b38f91062 service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:40:17.010556 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974a58f-e710-4587-b1a4-b87b38f91062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:40:17.012857 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-c974a58f-e710-4587-b1a4-b87b38f91062 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:40:17.025681 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974a58f-e710-4587-b1a4-b87b38f91062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:40:17.027158 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974a58f-e710-4587-b1a4-b87b38f91062 service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:40:17.037057 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1446/2898] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:40:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:40:20.763336 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fa5f6-6f81-4489-a844-93b7e573ca99 service nova] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:40:20.765043 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-ec3fa5f6-6f81-4489-a844-93b7e573ca99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:40:20.844181 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fa5f6-6f81-4489-a844-93b7e573ca99 service nova] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:40:20.849836 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afcc689-27f2-4ecd-8b47-ca2415b7ca51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '143c30b4201645279564e72ab3288adc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:40:20.850622 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcc689-27f2-4ecd-8b47-ca2415b7ca51 service placement] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:40:20.850968 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcc689-27f2-4ecd-8b47-ca2415b7ca51 service placement] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:40:20.851304 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afcc689-27f2-4ecd-8b47-ca2415b7ca51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:40:20.853299 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9afcc689-27f2-4ecd-8b47-ca2415b7ca51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:40:20.866366 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afcc689-27f2-4ecd-8b47-ca2415b7ca51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:40:20.868107 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afcc689-27f2-4ecd-8b47-ca2415b7ca51 service placement] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:40:20.879752 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1453/2899] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:40:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:40:27.032261 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=26 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:40:30.875902 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=32 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:40:31.692943 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974a58f-e710-4587-b1a4-b87b38f91062 service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:40:31.702482 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974a58f-e710-4587-b1a4-b87b38f91062 service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:40:31.706531 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b9796-d6ce-4219-a5a1-f9ec13840d8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '08229e9400b14e8a9cc8e8c32dc48285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:40:31.707138 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b9796-d6ce-4219-a5a1-f9ec13840d8f service neutron] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:40:31.707418 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b9796-d6ce-4219-a5a1-f9ec13840d8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:40:31.707689 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b9796-d6ce-4219-a5a1-f9ec13840d8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:40:31.709763 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-e15b9796-d6ce-4219-a5a1-f9ec13840d8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:40:31.778832 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b9796-d6ce-4219-a5a1-f9ec13840d8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:40:31.780564 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b9796-d6ce-4219-a5a1-f9ec13840d8f service neutron] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:40:31.794709 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1447/2900] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:40:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:40:34.741658 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afcc689-27f2-4ecd-8b47-ca2415b7ca51 service placement] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:40:34.744403 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-9afcc689-27f2-4ecd-8b47-ca2415b7ca51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:40:34.770681 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afcc689-27f2-4ecd-8b47-ca2415b7ca51 service placement] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:40:34.777450 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7481b63-57e6-420a-90ff-5ed06d283929 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '143c30b4201645279564e72ab3288adc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:40:34.778676 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7481b63-57e6-420a-90ff-5ed06d283929 service placement] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:40:34.779390 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7481b63-57e6-420a-90ff-5ed06d283929 service placement] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:40:34.780133 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7481b63-57e6-420a-90ff-5ed06d283929 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:40:34.782969 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f7481b63-57e6-420a-90ff-5ed06d283929 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:40:34.800628 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7481b63-57e6-420a-90ff-5ed06d283929 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:40:34.803482 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7481b63-57e6-420a-90ff-5ed06d283929 service placement] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:40:34.823873 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1454/2901] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:40:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 19:40:38.125125 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b9796-d6ce-4219-a5a1-f9ec13840d8f service neutron] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:40:38.136776 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b9796-d6ce-4219-a5a1-f9ec13840d8f service neutron] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:40:38.144202 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd3796-7854-4bde-8adc-c8c075e98670 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '143c30b4201645279564e72ab3288adc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:40:38.144914 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd3796-7854-4bde-8adc-c8c075e98670 service placement] REQUEST_METHOD: `GET` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:40:38.145279 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd3796-7854-4bde-8adc-c8c075e98670 service placement] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:40:38.145611 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd3796-7854-4bde-8adc-c8c075e98670 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:40:38.147510 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-6bfd3796-7854-4bde-8adc-c8c075e98670 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:40:38.160167 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd3796-7854-4bde-8adc-c8c075e98670 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:40:38.161961 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd3796-7854-4bde-8adc-c8c075e98670 service placement] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:40:38.180784 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1448/2902] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:40:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:40:38.194480 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7481b63-57e6-420a-90ff-5ed06d283929 service placement] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:40:38.196814 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f7481b63-57e6-420a-90ff-5ed06d283929 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:40:38.213858 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7481b63-57e6-420a-90ff-5ed06d283929 service placement] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:40:38.220248 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3661b64-6aca-4507-a4e6-2fbcf0fd05b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:40:38.221787 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3661b64-6aca-4507-a4e6-2fbcf0fd05b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:40:38.222078 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3661b64-6aca-4507-a4e6-2fbcf0fd05b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:40:38.222661 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3661b64-6aca-4507-a4e6-2fbcf0fd05b5 admin admin] PATH_INFO: `/v3/users/656a586e718543bfb095790441c28fe8` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:40:38.224089 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3661b64-6aca-4507-a4e6-2fbcf0fd05b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=656a586e718543bfb095790441c28fe8)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:40:38.226527 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3661b64-6aca-4507-a4e6-2fbcf0fd05b5 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:40:38.281849 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f3661b64-6aca-4507-a4e6-2fbcf0fd05b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '656a586e718543bfb095790441c28fe8'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:38.298353 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f3661b64-6aca-4507-a4e6-2fbcf0fd05b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '656a586e718543bfb095790441c28fe8'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:38.309191 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f3661b64-6aca-4507-a4e6-2fbcf0fd05b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '656a586e718543bfb095790441c28fe8'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:38.309947 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f3661b64-6aca-4507-a4e6-2fbcf0fd05b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '656a586e718543bfb095790441c28fe8'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:38.310547 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-f3661b64-6aca-4507-a4e6-2fbcf0fd05b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '656a586e718543bfb095790441c28fe8'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:38.329305 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1455/2903] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:40:38 2022] DELETE /identity/v3/users/656a586e718543bfb095790441c28fe8 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:40:38.515302 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd3796-7854-4bde-8adc-c8c075e98670 service placement] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:40:38.527817 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd3796-7854-4bde-8adc-c8c075e98670 service placement] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:40:38.588939 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8cd04-311d-4717-97fc-adc4e415df12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:40:38.589591 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8cd04-311d-4717-97fc-adc4e415df12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:40:38.589873 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8cd04-311d-4717-97fc-adc4e415df12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:40:38.590148 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8cd04-311d-4717-97fc-adc4e415df12 admin admin] PATH_INFO: `/v3/projects/bdc0726934c64051b49ed81a2abc5c89` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:40:38.591376 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8cd04-311d-4717-97fc-adc4e415df12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc0726934c64051b49ed81a2abc5c89)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:40:38.593388 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8cd04-311d-4717-97fc-adc4e415df12 admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:40:38.649820 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-aeb8cd04-311d-4717-97fc-adc4e415df12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc0726934c64051b49ed81a2abc5c89', 'request_id': 'req-aeb8cd04-311d-4717-97fc-adc4e415df12'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:38.656790 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-aeb8cd04-311d-4717-97fc-adc4e415df12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc0726934c64051b49ed81a2abc5c89', 'request_id': 'req-aeb8cd04-311d-4717-97fc-adc4e415df12'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:38.664472 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-aeb8cd04-311d-4717-97fc-adc4e415df12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc0726934c64051b49ed81a2abc5c89', 'request_id': 'req-aeb8cd04-311d-4717-97fc-adc4e415df12'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:38.679975 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-aeb8cd04-311d-4717-97fc-adc4e415df12 admin admin] The token cache is being invalidate because project bdc0726934c64051b49ed81a2abc5c89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:40:38.680278 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-aeb8cd04-311d-4717-97fc-adc4e415df12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:38.680919 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-aeb8cd04-311d-4717-97fc-adc4e415df12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:38.682438 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1449/2904] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:40:38 2022] DELETE /identity/v3/projects/bdc0726934c64051b49ed81a2abc5c89 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:40:38.686937 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3661b64-6aca-4507-a4e6-2fbcf0fd05b5 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:40:38.688501 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-f3661b64-6aca-4507-a4e6-2fbcf0fd05b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:40:38.702697 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3661b64-6aca-4507-a4e6-2fbcf0fd05b5 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:40:38.771561 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9b18b-374f-4812-88a2-359c67781881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:40:38.772261 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9b18b-374f-4812-88a2-359c67781881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:40:38.772546 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9b18b-374f-4812-88a2-359c67781881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:40:38.772821 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9b18b-374f-4812-88a2-359c67781881 admin admin] PATH_INFO: `/v3/users/e58ea4f0b3414240ab7eb521e7b92db1` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:40:38.773974 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9b18b-374f-4812-88a2-359c67781881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e58ea4f0b3414240ab7eb521e7b92db1)` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:40:38.775945 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9b18b-374f-4812-88a2-359c67781881 admin admin] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:40:38.824431 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-4dd9b18b-374f-4812-88a2-359c67781881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e58ea4f0b3414240ab7eb521e7b92db1'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:38.834767 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:40:38.837192 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-4dd9b18b-374f-4812-88a2-359c67781881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e58ea4f0b3414240ab7eb521e7b92db1'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:38.845697 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-4dd9b18b-374f-4812-88a2-359c67781881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e58ea4f0b3414240ab7eb521e7b92db1'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:38.846975 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-4dd9b18b-374f-4812-88a2-359c67781881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e58ea4f0b3414240ab7eb521e7b92db1'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:38.847634 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.notifications [None req-4dd9b18b-374f-4812-88a2-359c67781881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e58ea4f0b3414240ab7eb521e7b92db1'} {{(pid=80139) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:38.865707 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1456/2905] 10.222.0.17 () {58 vars in 1224 bytes} [Sun Aug 7 19:40:38 2022] DELETE /identity/v3/users/e58ea4f0b3414240ab7eb521e7b92db1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:40:39.047249 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8cd04-311d-4717-97fc-adc4e415df12 admin admin] Authenticating user token {{(pid=80138) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:40:39.049171 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.fernet_utils [None req-aeb8cd04-311d-4717-97fc-adc4e415df12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80138) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:40:39.067909 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8cd04-311d-4717-97fc-adc4e415df12 admin admin] Validating token access rules against request {{(pid=80138) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:40:39.078523 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:40:39.132740 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0861afd4-89a5-4b4e-b455-e8bd49565a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '39e61fe22f50447d975033301fd5b706', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c00a4064cc8477492c9b8b2e5412197', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80138) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:40:39.133498 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861afd4-89a5-4b4e-b455-e8bd49565a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:40:39.133672 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861afd4-89a5-4b4e-b455-e8bd49565a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:40:39.133940 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0861afd4-89a5-4b4e-b455-e8bd49565a1d admin admin] PATH_INFO: `/v3/projects/ac61655af79e4fcf9c54979977957d2e` {{(pid=80138) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:40:39.135076 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861afd4-89a5-4b4e-b455-e8bd49565a1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac61655af79e4fcf9c54979977957d2e)` {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:40:39.137215 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0861afd4-89a5-4b4e-b455-e8bd49565a1d admin admin] RBAC: Authorization granted {{(pid=80138) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:40:39.196427 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0861afd4-89a5-4b4e-b455-e8bd49565a1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac61655af79e4fcf9c54979977957d2e', 'request_id': 'req-0861afd4-89a5-4b4e-b455-e8bd49565a1d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:39.202571 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0861afd4-89a5-4b4e-b455-e8bd49565a1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac61655af79e4fcf9c54979977957d2e', 'request_id': 'req-0861afd4-89a5-4b4e-b455-e8bd49565a1d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:39.209673 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0861afd4-89a5-4b4e-b455-e8bd49565a1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac61655af79e4fcf9c54979977957d2e', 'request_id': 'req-0861afd4-89a5-4b4e-b455-e8bd49565a1d'} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:39.224287 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0861afd4-89a5-4b4e-b455-e8bd49565a1d admin admin] The token cache is being invalidate because project ac61655af79e4fcf9c54979977957d2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80138) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 19:40:39.224593 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0861afd4-89a5-4b4e-b455-e8bd49565a1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:39.225211 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG keystone.notifications [None req-0861afd4-89a5-4b4e-b455-e8bd49565a1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80138) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 19:40:39.226712 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: [pid: 80138|app: 0|req: 1450/2906] 10.222.0.17 () {58 vars in 1233 bytes} [Sun Aug 7 19:40:39 2022] DELETE /identity/v3/projects/ac61655af79e4fcf9c54979977957d2e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 19:40:48.860427 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:40:49.220859 dv-cinder-850320-4-iscsi devstack@keystone.service[80138]: DEBUG dbcounter [-] [80138] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80138) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 19:45:27.674934 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9b18b-374f-4812-88a2-359c67781881 admin admin] Authenticating user token {{(pid=80139) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 19:45:27.676488 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-4dd9b18b-374f-4812-88a2-359c67781881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:45:27.829633 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9b18b-374f-4812-88a2-359c67781881 admin admin] Validating token access rules against request {{(pid=80139) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 19:45:27.893637 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529600c5-4308-4a6a-b79d-64e500a72b9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '143c30b4201645279564e72ab3288adc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '37703d50cd7e4d9bbdc797a7a1466b66', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80139) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 19:45:27.894258 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529600c5-4308-4a6a-b79d-64e500a72b9b service placement] REQUEST_METHOD: `GET` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 19:45:27.894542 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529600c5-4308-4a6a-b79d-64e500a72b9b service placement] SCRIPT_NAME: `/identity` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 19:45:27.894817 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529600c5-4308-4a6a-b79d-64e500a72b9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80139) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 19:45:27.896786 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.fernet_utils [None req-529600c5-4308-4a6a-b79d-64e500a72b9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80139) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 19:45:27.908888 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529600c5-4308-4a6a-b79d-64e500a72b9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 19:45:27.910491 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529600c5-4308-4a6a-b79d-64e500a72b9b service placement] RBAC: Authorization granted {{(pid=80139) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 19:45:27.921830 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: [pid: 80139|app: 0|req: 1457/2907] 10.222.0.17 () {60 vars in 1365 bytes} [Sun Aug 7 19:45:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 19:45:37.916966 dv-cinder-850320-4-iscsi devstack@keystone.service[80139]: DEBUG dbcounter [-] [80139] Writing DB stats keystone:SELECT=89 {{(pid=80139) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}